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(54) Apparatus for displaying image recognized by observer as stereoscopic image, and image 
pick-up apparatus 



(57) This invention relates to a stereoscopic image 
display apparatus which allows an observer to recog- 
nize a stereoscopic image, comprising right eye image 
display member for displaying an image for a right eye 
by irradiating light, left eye image display member for 
displaying an image for a left eye by irradiating light, and 
tilt member for tilting a centra I line, in a vertical direction, 
of an observation image which is formed based on the 
image for the right eye and is to be observed by the right 
eye of the observer, and a central line, in the vertical 
direction, of an observation image which is formed 
based on the image for the left eye and is to be 
observed by the left eye of the observer, relative to each 
other within a range from an angle larger than 0° to an 
angle not more than 3°. 
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Description 

BACKGROUND OF THE INVENTION 
5 Field of the Invention 

[0001] The present invention relates to an image display apparatus and an image recording apparatus and, more 
particularly, to a technique suitable for an apparatus for displaying display images on image display means on the basis 
of a plurality of input images with a parallax, so that an observer visually recognizes a plurality of observation images 
10 formed by the display images with his or her eyes, and recognizes them as a stereoscopic image. 

Related Background Art 

[0002] Conventionally, many image display apparatuses which independently present a plurality of images with a 
15 parallax to the two eyes of an observer so as to cause the observer to recognize a stereoscopic image have been pro- 
posed. Chihiro Masuda, "Three-dimensional Display" (Sangyo Tosho) discloses various types of stereoscopic image 
display apparatuses, as will be described below. 

[0003] For the sake of easy understanding, images observed by the two eyes of an observer will be referred to as 
"observation images* hereinafter, images fetched from an object by input means will be referred to as "input images" 
20 hereinafter, and images displayed on image display means for the purpose of observation will be referred to as "display 
images" hereinafter. 

[0004] Fig. 1 is an explanatory view of a three-dimensional display (stereoscopic image display apparatus) of an 
anaglyph method. In this method, a display image for the right eye and a display image for the left eye are respectively 
super-impose-displayed in two colors, e.g., red and blue, and observation images for the right and left eyes are sepa- 
25 rated using color filters, thus allowing an observer to recognize a stereoscopic image. 

[0005] Fig. 2 is an explanatory view of a three-dimensional display of a polarizing glass method. In this method, a 
display image for the right eye and a display image for the left eye are respectively displayed on CRTs. Polarizing filters 
(polarizing plates) having orthogonal planes of oscillation and a half mirror are arranged in front of the CRTs. Observa- 
tion images for the right and left eyes are separated by polarizing glasses, thus allowing an observer to recognize a ster- 
eo eoscopic image. 

[0006] Fig. 3 is an explanatory view of a three-dimensional display of a time sharing shutter method. In this method, 
display images for the right and left eyes are alternately displayed on a CRT in a time sharing manner, and observation 
images for the right and left eyes are separated using time sharing shutter glasses which are opened/closed in a time 
sharing manner in synchronism with the display images, thus allowing an observer to recognize a stereoscopic image. 
35 [0007] Fig. 4 is an explanatory view of an optical method. In this method, display images for the right and left eyes, 
which are separately displayed on a viewer using optical means such as prisms, mirror, lens, and the like, are superim- 
pose-displayed as observation images in front of an observer, thus allowing the observer to recognize a stereoscopic 
image. 

[0008] Fig. 5 is an explanatory view of a three-dimensional display of a lenticular method. In this method, a plurality 
40 of input images input by a lenticular lens are separately displayed via corresponding lenticular lens portions, thus allow- 
ing a plurality of observers to recognize a stereoscopic image. 

[0009] Figs. 6A and 6B are explanatory views of a three-dimensional display of large convex lens and large con- 
cave mirror methods. Fig. 6A shows the principle of the large convex lens method, and Fig. 6B shows the principle of 
the large concave mirror method. In this method, display images for the right and left eyes are formed on a large convex 

45 lens or a large concave mirror using projectors, and are displayed while determining the setting positions of the projec- 
tors and the spatial positions of the display images for the right and left eyes, so that light rays from the corresponding 
display images are incident on the right and left eyes, thus allowing an observer to recognize a stereoscopic image. 
[0010] Fig. 7 is an explanatory view of a three-dimensional display of a parallax barrier method. In this method, a 
slit-shaped barrier (parallax barrier) is disposed in front of image display means to split a display image into right and 

so left display images, and the right and left visual axes recognize these display images as observation images via the bar- 
rier, thus allowing an observer to recognize a stereoscopic image. 

[0011] Figs. 8A, 8B, and 8C are explanatory views of a three-dimensional display of an integral method. In this 
method, a large number of parallax images of an object are received (input) and recorded by an image taking device 
via a fly eye lens (Fig. 8A), and are transferred to a display device to be displayed as an image on the display device. 
55 The respective parallax images spatially form a real image at an identical position as that in the image taking operation. 
When an observer observes this real image from various directions, he or she can observe a parallax image corre- 
sponding to the observation direction, thus allowing stereoscopic viewing (Figs. 8B and 8C). 

[0012] On the other hand, as an input method of input images with a parallax, which are used in each of the above- 
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mentioned image display apparatuses, a method of taking an image of an object from a plurality of directions using a 
plurality of cameras is normally used. When only two input images with a parallax or the right and left eyes are required 
like in the anaglyph method, polarizing glass method, time sharing shutter method, optical method, and the like, two 
image taking cameras Y R and Y L need only be used for an object X to be taken, as shown in Fig. 9, so as to obtain input 
5 images Xr and X L . 

[0013] However, the lenticular method,' large convex lens/large concave mirror method, parallax barrier method, 
and the like require input images with a parallax from three or more view points in some cases. Thus, such methods 
require multi-lens camera input means, as shown in Fig. 25 (reference numerals in Fig. 25 will be described in detail 
later). 

w [0014] Note that the integral method inputs images via a special-purpose fly eye lens. 

[001 5] On the other hand, in recent years, input images actually taken by cameras are subjected to image process- 
ing attained by arithmetic operations of a computer so as to obtain parallax images for stereoscopic viewing, and such 
images are often used in stereoscopic viewing. In some cases, images of an object processed to appear to have been 
taken from quite different directions are generated and input on the basis of several input images, and such images are 

15 often used in the above-mentioned stereoscopic image display apparatus. 

[0016] Furthermore, input images with a parallax, which are called CG (computer graphics) and are artificially syn- 
thesized by arithmetic operations using a computer, are often used in place of input images actually taken by cameras. 
When a CG method which performs image generation by accurately recognizing object configuration information in a 
three-dimensional space is used, since a plurality of input images with a parallax can be formed on the basis of single 
20 three-dimensional information, such a method can be easily applied to the stereoscopic image display apparatus. 

[0017] The above-mentioned stereoscopic image display utilizing a binocular parallax has no established display 
condition between the observation images for the right and left eyes, but is premised on the simple assumption that 
images for the right and left eyes are displayed under an identical condition. No image display apparatus which sets the 
display condition for images for the right and left eyes in consideration of effects to be given to an observer is available. 
25 [0018] Similarly, as for the image input apparatus for inputting images to the image display apparatus for displaying 
a stereoscopic image, no apparatus which sets the display condition for images for the right and left eyes in considera- 
tion of effects to be given to an observer is available. 

SUMMARY OF THE INVENTION 

30 

[0019] It is a concern of the present invention to provide an image display apparatus and an image input apparatus, 
which can improve observability by setting different display conditions of images to be presented to the right and left 
eyes of an observer. 

[0020] According to one aspect of the present invention, there is provided a stereoscopic image display apparatus 
35 which allows an observer to recognize a stereoscopic image, comprising: 

right eye image display means for displaying an image for a right eye by irradiating light; 
left eye image display means for displaying an image for a left eye by irradiating light; and 

tilt means for tilting a central line, in a vertical direction, of an observation image which is formed based on the 
image for the right eye and is to be observed by the right eye of the observer, and a central line, in the vertical direc- 
tion, of an observation image which is formed based on the image for the left eye and is to be observed by the left 
eye of the observer, relative to each other within a range from an angle larger than 0° to an angle not more than 3°. 

[0021] According to a preferred aspect, the tilt means comprises means for rotating at least one of the right and left 
45 eye image display means in a display surface of the image display means. 

[0022] According to a preferred aspect, the tilt means comprises an image rotator arranged in an optical path of at 
least one of the right and left eye image display means. 

[0023] According to a preferred aspect, the tilt means comprises means for rotating a display image of at least one 
of the right and left eye image display means by coordinate conversion processing, and displaying the rotated image on 
so the image display means. 

[0024] According to a preferred aspect, the tilt means comprises one of means for rotating clockwise the central 
line, in the vertical direction, of the observation image to be observed by the right eye of the observer and means for 
rotating counterclockwise the central line, in the vertical direction, of the observation image to be observed by the left 
eye of the observer. 



40 



55 



[0025] According to one aspect of the present invention, there is provided an image recording apparatus for record- 
ing an image which is used to allow an observer to recognize a stereoscopic image, comprising: 

right eye image recording means for taking an input image for a right eye of the observer, and recording the input 
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image; 

left eye image recording means for taking an input image for a left eye of the observer, and recording the input 
image; and 

rotation means for rotating at least one of the right and left eye image recording means in a recording surface of the 
5 image recording means so as to tilt and record the input images for the right and left eyes relative to each other. 

[0026] According to a preferred aspect, the rotation means comprises an image rotator arranged in an optical path 
upon taking one of the input images for the right and left eyes. 

[0027] According to another aspect of the present invention, there is provided a stereoscopic image display appa- 
io ratus which allows an observer to recognize a stereoscopic image, comprising: 

right eye image display means for displaying an image for a right eye by irradiating light; 
left eye image display means for displaying an image for a left eye by irradiating light; and 

means for setting a size of an observation image which is formed based on the image for the right eye and is to be 
is observed by the right eye of the observer, and a size of an observation image which is formed based on the image 

for the left eye and is to be observed by the left eye of the observer to be relatively different from each other. 

[0028] According to a preferred aspect, the means for setting the sizes of the observation images to be relatively 
different from each other comprises means for changing a size of a display image of at least one of the right and left 
20 eye image display means by coordinate conversion means, and displaying the image of the changed size on the image 
display means. 

[0029] According to a preferred aspect, the image display apparatus further comprises a right eye optical system 
for forming the observation image to be observed by the right eye of the observer on the basis of the image for the right 
eye, and a left eye optical system for forming the observation image to be observed by the left eye of the observer on 
25 the basis of the image for the left eye, and a focal length of the right eye optical system is different from a focal length 
of the left eye optical system. 

[0030] According to a preferred aspect, the focal length of at least one of the right and left eye optical systems is 
variable. 

[0031] According to a preferred aspect, the stereoscopic image display apparatus satisfies: 

30 

0 < IAVI < 3 

where R v and L v are the sizes of the observation images to be observed by the right and left eyes of the observer, and 
AV is the magnification difference given by: 

35 

AV = (R V -L V )/L/100 

[0032] R v and L v are preselected measuring lengths of virtual images to be objected in this specification. 
[0033] According to the present invention, there is provided an image pick-up apparatus for taking an image which 
40 is used to allow an observer to recognize a stereoscopic image, comprising: 

a right eye image pick-up optical system for taking an image for a right eye of the observer; and 
a left eye image pick-up optical system for taking an image for a left eye of the observer, 
wherein the right and left eye image pick-up optical systems have different magnification factors. 

45 

[0034] According to a preferred aspect, the magnification factor of at least one of the right left eye image pick-up 
optical systems is variable. 

[0035] According to a preferred aspect, the image pick-up apparatus satisfies: 

50 0 < IAVI < 3 

where AV is the magnification difference between the right and left image pick-up optical systems. 

[0036] Some examples of an image display apparatus of the present invention will be presented in the following 

description of the embodiments. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
[0037] 

Fig. 1 is an explanatory view of a three-dimensional display of an anaglyph method: 
Fig. 2 is an explanatory view of a three-dimensional display of a polarizing glass method; 
Fig. 3 is an explanatory view of a three-dimensional display of a time sharing shutter method; 
Fig. 4 is an explanatory view of a three-dimensional display of an optical method; 
Fig. 5 is an explanatory view of a three-dimensional display of a lenticular method; 

Figs. 6A and 6B are explanatory views of three-dimensional displays of large convex lens and large concave mirror 
methods; 

Fig. 7 is an explanatory view of a three-dimensional display of a parallax method; 

Figs. 8A, 8B, and 8C are explanatory views of a three-dimensional display of an integral method; 

Fig. 9 is a view showing the image pick-up layout of two input images with a parallax; 

Fig. 10 is an explanatory view of the central line, L, in the vertical direction of an observation image; 

Figs. 1 1 A and 1 1 B are experiment graphs I showing the relationship between the difference in display condition and 

the observability of an observation image; 

Figs. 12A and 12B are experiment graphs II showing the relationship between the difference in display condition 
and the observability of an observation image; 

Fig. 13 is a schematic view showing principal part of the first embodiment of the present invention; 

Figs. 1 4A and 14B are front views showing the layout of image displays of the first embodiment, in which Fig. 1 4A 

shows the right image display, and Fig. 14B shows the left image display; 

Figs. 15A and 15B are views of observation images of the prior art and the first embodiment when observed from 

an observer, in which Fig. 1 5A shows the prior art, and Fig. 1 5B shows the first embodiment; 

Fig. 16 is a schematic view showing principal part of the second embodiment of the present invention; 

Fig. 1 7 is a schematic view showing principal part of the third embodiment of the present invention; 

Figs. 18A and 18B are explanatory views of an image signal modulating portion in the third embodiment, in which 

Fig. 18A shows the arrangement for an analog image signal, and Fig. 18B shows the arrangement for a digital 

image signal; 

Fig. 19 is a view showing display images displayed on the right and left image displays of the third embodiment; 
Fig. 20 is a schematic view showing principal part of the fourth embodiment of the present invention; 
Figs. 21 A and 21 B are perspective views of an image rotator; 

Fig. 22 is a schematic view showing principal part of the fifth embodiment of the present invention; 

Fig. 23 is a schematic view showing principal part of the sixth embodiment of the present invention; 

Fig. 24 is a schematic view showing principal part of another arrangement of the sixth embodiment of the present 

invention; 

Fig. 25 is an explanatory view of six-lens camera input means; 

Fig. 26 is an explanatory view of an observation method of images obtained by a multi-lens camera system; 
Fig. 27 is a schematic view showing principal part of the seventh embodiment of the present invention; 
Fig. 28 is a schematic view showing principal part of another arrangement of the seventh embodiment of the 
present invention; 

Fig. 29 is a schematic view showing principal part of the eighth embodiment of the present invention; 
Fig. 30 is an explanatory view of generation of input images from virtual view points; 
Fig. 31 is a schematic view showing principal part of the ninth embodiment of the present invention; 
Fig. 32 is an explanatory view of the general procedure of stereoscopic image display by the anaglyph method; 
Fig. 33 is a perspective view showing a two-image projection type image display apparatus for an anaglyph stere- 
oscopic image; 

Fig. 34 is an explanatory view of the general procedure of stereoscopic image display by the anaglyph method 
shown in Fig. 31; 

Fig. 35 is an explanatory view of the general procedure of stereoscopic image display by the polarizing glass 
method; 

Fig. 36 is an explanatory view of the general procedure of stereoscopic image display by the time sharing shutter 
method; 

Fig. 37 is an explanatory view of the general procedure of stereoscopic image display by the large convex 
lens/large concave mirror method; 

Fig. 38 is an explanatory view of the general procedure of stereoscopic image display by the lenticular method; 
Fig. 39 is an explanatory view of stereoscopic image display of the lenticular method by means of image projection; 
Fig. 40 is an explanatory view of the general procedure of stereoscopic image display of the lenticular method 
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based on image projection; 

Fig. 41 is an explanatory view of the general procedure of stereoscopic image display by the parallax barrier 
method; 

Figs. 42A and 42B are graphs showing the relationship between the magnification difference and observability; 
Fig. 43 is a schematic view showing principal part of the 10th embodiment of the present invention; 
Fig. 44 is a schematic view showing principal part of a modification of Fig. 43; 
Fig. 45 is a schematic view showing principal part of the 1 1th embodiment of the present invention; 
Fig. 46 is an explanatory view of an image signal modulating portion when an image signal is an analog signal; 
Fig. 47 is an explanatory view of an image signal modulating portion when an image signal is a digital signal; 
Figs. 48A and 48B are explanatory views of display images upon setting a magnification difference; 
Figs. 49A to 49D are explanatory views of the size of an image; 

Fig. 50 is a schematic view showing principal part of the 12th embodiment of the present invention. 
Fig. 51 is an explanatory view of the positional relationship among the respective elements in the 12th embodiment; 
Fig. 52 is a schematic view showing principal part of the 13th embodiment of the present invention; 
Fig. 53 is a schematic view showing principal part of the 14th embodiment of the present invention; 
Fig. 54 is a schematic view showing principal part of the 14th embodiment of the present invention; 
Fig. 55 is a schematic view showing principal part of the 14th embodiment of the present invention; 
Fig. 56 is a schematic view showing principal part of the 14th embodiment of the present invention- 
Fig. 57 is a schematic view showing principal part of the 15th embodiment of the present invention; 
Fig. 58 is an explanatory view upon displaying taken images obtained in the 15th embodiment on an image display 
apparatus; 

Fig. 59 is a schematic view showing principal part of the 15th embodiment of the present invention; 
Fig. 60 is a schematic view showing principal part of the 15th embodiment of the present invention- 
Fig. 61 is a schematic view showing principal part of the 1 6th embodiment of the present invention- 
's Fig. 62 is a schematic view showing principal part of the 16th embodiment of the present invention- 
Fig. 63 is a schematic view showing principal part of the 17th embodiment of the present invention- 
Fig. 64 is an explanatory view upon displaying taken images obtained in the 1 7th embodiment on an image display 
apparatus; 

Figs. 65A and 65B are schematic views showing principal part of the 18th embodiment of the present invention; 
30 Fig. 66 is an explanatory view of the 1 9th embodiment of the present invention; 

Fig. 67 is a schematic view showing principal part of the 19th embodiment of the present invention; 

Fig. 68 is an explanatory view of the 1 9th embodiment of the present invention; 

Rg. 69 is a schematic view showing principal part of the 20th embodiment of the present invention; 

Fig. 70 is an explanatory view of the 21 st embodiment of the present invention; 
35 Rg. 71 is an explanatory view of the 22nd embodiment of the present invention; 

Fig. 72 is an explanatory view of the 23rd embodiment of the present invention; 

Fig. 73 is a schematic view showing principal part of the 23rd embodiment of the present invention; 

Fig. 74 is an explanatory view of the 23rd embodiment of the present invention; 

Fig. 75 is an explanatory view of the 24th embodiment of the present invention; and 
40 Rg. 76 is an explanatory view of the 25th embodiment of the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
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[0038] The conventional stereoscopic image display utilizing the above-mentioned binocular parallax described 
above has no established display condition between the observation images for the right and left eyes, but is premised 
on the simple assumption that observation images for the right and left eyes are displayed so that their central lines in 
the vertical direction always extend parallel to each other. Note that the central line, in the vertical direction, of the obser- 
vation image means a straight line L in Fig. 10, i.e., a straight line which perpendicularly crosses the horizontal line in 
the display plane of the observation image and passes the center of the image. 

[0039] No image display apparatus which sets the display condition for images for the right and left eyes in consid- 
eration of effects to be given to an observer when the above-mentioned display condition is changed is available. 
[0040] Also, there is no proposal associated with input means of parallax images used in an image display appara- 
tus for displaying a stereoscopic image by setting a different display condition from that of the conventional apparatus. 
[0041] As a result of examination of the effects of different display conditions between observation images for the 
right and left eyes on an observer in the stereoscopic image display utilizing a binocular parallax, the present inventors 
found that the "observability" of a stereoscopic image could be improved by displaying observation images for the right 
and left eyes while their central lines in the vertical direction were tilted in a V-shaped pattern relative to each other. 
[0042] This effect will be explained below. Figs. 1 1 A and 1 1 B and Figs. 12A and 12B are graphs showing the exam- 
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ination results. In each of these graphs, the rotational angle (degrees) of the observation image is plotted along the 
abscissa, and the evaluation value statistically obtained by distributed processing of subjective evaluation values of 
"observability" of a stereoscopic image for a plurality of observers is plotted along the ordinate. Note that the "observa- 
bility" represents an evaluation pertaining to "the degree of fatigue after observation of a stereoscopic image for a long 
5 period of time". 

[0043] Fig. 1 1 A shows a case wherein the observation image for the left eye is fixed in position, and the observation 
image for the right eye is rotated clockwise, and Fig. 1 1 B shows a case wherein the observation image for the right eye 
is fixed in position, and the observation image for the left eye is rotated clockwise. As can be seen from Fig. 1 1 B, when 
the observation image for the right eye is fixed in position, and the observation image for the left eye is rotated clock- 

w wise, the evaluation value of "observability" of a stereoscopic image lowers as the rotational angle increases. In con- 
trast, as can be seen from Fig. 1 1 A, when the observation image for the left eye is fixed in position, and the observation 
image for the right eye is rotated clockwise, the evaluation value at "observability" of a stereoscopic image becomes 
higher upon rotation through about 2° than that obtained when the rotational angle of the image = 0°. 
[0044] Also, as can be seen from Fig. 1 1 A, the evaluation value of "observability" of a stereoscopic image continues 

15 to improve within the range from 0° to 3°. 

[0045] Fig. 12A shows a case wherein the observation image for the right eye is fixed in position, and the observa- 
tion image for the left eye is rotated counterclockwise, and Fig. 12B shows a case wherein the observation image for 
the left eye is fixed in position, and the observation image for the right eye is rotated counterclockwise. As can be seen 
from Fig. 12A, the evaluation value of "observability" of a stereoscopic image is maintained when the rotational angle 

20 of the image = 0° and when the image is rotated through about 2°. However, as can be seen from Fig. 12B, when the 
observation image for the left eye is fixed in position, and the observation image for the right eye is rotated counter- 
clockwise, the evaluation value of "observability" of a stereoscopic image lowers as the rotational angle increases. 
[0046] As can be seen from the above description, when the observation images for the right and left eyes are dis- 
played to be tilted relative to each other in their display planes, the evaluation value of "observability" of a stereoscopic 

25 image to be observed changes, and the stereoscopic image can become more "observable" than the conventional dis- 
play method, i.e., when the rotational angle = 0° depending on the direction of rotation. 

[0047] However, none of conventional stereoscopic image display apparatuses adopt a positive device for rotating 
the observation images for the right and left eyes relative to each other in their display planes. Similarly, none of image 
input means of the conventional image display apparatuses adopt a positive device for rotating input images to be input 

30 for the right and left eyes relative to each other. 

[0046] It is an object of the present invention to provide an image display apparatus and an image recording appa- 
ratus, which can present a better and more "observable" stereoscopic image than a conventional stereoscopic image 
display apparatus, i.e., can obtain a state wherein "an observer does not experience fatigue after observation of a ster- 
eoscopic image for a long period of times". 

35 [0049] Furthermore, it is another object of the present invention to provide an image display apparatus and an 
image recording apparatus, which can arbitrarily adjust the "observability" of a stereoscopic image within the above- 
mentioned range in correspondence with the recognition ability of each observer. 

[0050] Fig. 13 is a schematic view showing principal part of the first embodiment of the present invention. In this 
embodiment, the present invention is applied to a so-called HMD (head-mounted display) type image display apparatus 

40 which independently presents two observation images with a parallax to the two eyes of an observer via an optical sys- 
tem to allow the observer to recognize a stereoscopic image. Referring to Fig. 13, image displays (image display 
means) 1 R and 1 L (subscripts R and L respectively indicate elements for the right and left eyes) comprise liquid crystal 
display elements (to be abbreviated as LCDs hereinafter) with backlights in this embodiment. As the image display, a 
CRT, a plasma display, or the like may be used. In order to attain a size reduction of the entire apparatus, a compact 

45 display is preferably used. 

[0051] An image signal generating portion 2 generates image signals of display images 4 R and 4 L to be respec- 
tively presented to the right and left eyes of an observer, and transmits these signals to the corresponding image dis- 
plays. Lenses (optical systems) 3 R and 3 L have a role of forming, in an enlarged scale, virtual images (observation 
images) 4 n ' and 4 L ' of the display images 4 R and 4l displayed on the image displays 1 at positions in front of the 

50 observer. Mirrors 5 R and 5 L bend the optical axes of the lenses 3 n and 3 L . Eyes 6 R and 6 L of the observer are located 
on the optical axes bent by the mirrors 5 R and 5 L , and the observer can observe the observation images 4 R and 4 L ' 
enlarged in the optical axis directions. Since the mirrors 5 R and 5 L are used, the apparatus can be made compact, and 
can be mounted on the head of the observer. When half mirrors are used, a background image can be superposed on 
the observation images 4 R ' and 4 L \ and when the mirrors 5 themselves have an optical power, a further size reduction 

55 and improvement of optical performance can be attained. 

[0052] Note that optical systems 30 R and 30 L for the right and left eyes, which are respectively constituted by the 
mirrors 3 R and 3 L and the mirrors 5 R and 5 L , are disposed at substantially symmetrical positions about the vertical 
direction, and form the observation images 4 R and 4 L ' at substantially the same position in a space, as shown in Fig. 



BNSDCCID: <EP 1 083464 A2J„ 



10 



15 



EP 1 083 454 A2 

13. For this reason, when the display images 4 R and 4 L having a binocular parallax are respectively displayed on the 
right and left image displays 1 R and 1 L , the observer can easily observe a stereoscopic image. 

[0053] In this embodiment, the image displays 1 R and 1 L are disposed so that the bottom sides of their screen 
frames do not extend horizontally, unlike in the conventional image display apparatus. Fig. 14A shows the right image 
display 1 R when viewed from the display plane side (from the observer side) via the mirror. In Fig. 14A, the layout of the 
conventional image display is indicated by a dotted line, and the layout of the image display of this embodiment is indi- 
cated by a solid line. As shown in Fig. 14A, the right image display 1 R is rotated clockwise through about 1° from the 
layout of the conventional image display in its display plane. 

[0054] On the other hand, the left image display 1 L is rotated counterclockwise through about 1 ° from the conven- 
tional layout in its display plane, as shown in Fig 14B. More specifically, the central lines, in the vertical directions of the 
observation images forthe right and left eyes are tilted in a V-shaped pattern. (In the following description, the V-shaped 
pattern represents a state wherein the central lines, in the vertical direction, of two images are titled in a V-shaped pat- 
tern relative to each other.) With this layout, the observation images 4 R ' and 4 L ' are also formed in the directions of the 
corresponding displays. 

[0055] Fig. 15A shows the observation images 4 R ' and 4 L ' of the conventional display apparatus when viewed from 
the direction of the observer. The bottom sides of the right and left observation images are horizontal, and the positions 
of the entire right and left image frames substantially match each other. However, in this embodiment, as shown in Fig. 
15B, the observation images 4 R . and 4 L ' for the right and left eyes are formed in a state wherein the observation image 
4 R ' for the right eye is rotated clockwise through about 1°, and the observation image 4 L ' for the left eye is rotated coun- 
20 terclockwise through about 1 °, i.e., the two images are tilted through 2° relative to each other. 

[0056] As described above, in the HMD type stereoscopic image display apparatus of this embodiment, the two 
observation images with a parallax, which are presented to the two images, are tilted relative to each other within the 
range from an angle larger than 0° to an angle equal to or smaller than 3°, and preferably, within the range from 1° to 
3°, thus obtaining good "observability" of a stereoscopic image. 
25 [0057] In the present invention, since a relative tilt angle need only be set between the two observation images one 
of the two image displays may be tilted from the horizontal state. 

[0058] Fig. 1 6 is a schematic view showing principal part of the second embodiment of the present invention. In this 
embodiment, the mirrors 5 in the first embodiment are omitted. Eyes 6 R and 6 L of the observer are located on the opti- 
cal axes of the lenses 3 R and 3 L , and the observer can observe the virtual images (observation images) 4 R ' and 4,' 

so enlarged in the optical axis directions. Optical systems 31 R and 31 L for the right and left eyes, which are respectively 
constituted by the image displays 1 R and 1 L and the lenses 3 R and 3 L , are disposed at substantially the symmetrical 
positions about the vertical direction, and form the observation images 4 R ' and 4 L ' at substantially the same position on 
the space, as shown in Fig. 16. For this reason, when the display images 4 R and 4 L having a binocular parallax are 
respectively displayed on the right and left image displays 1 R and 1 L , the observer can easily observe a stereoscopic 

35 image. 

[0059] In this arrangement as well, the right and left image displays 1 R and 1 L are disposed in a tilted state as in 
the first embodiment. With this arrangement, the observation images 4 R ' and 4 L ' are tilted in a V-shaped pattern relative 
to each other within the range from an angle larger than 0° to an angle equal to or smaller than 3°, and preferably, within 
the range from 1 ° to 3°, thus obtaining good "observability" of a stereoscopic image. 
40 [0060] As described above, the present invention is characterized in that the right and left observation images to be 
observed by the observer are displayed to be tilted relative to each other in their display planes. As the rotation method 
of the two observation images, the following two methods are available in addition to the method of tilting the image dis- 
plays. 

45 1 ) Method using coordinate conversion of images 
2) Method using optical systems 

[0061] Method 1) using coordinate conversion of images will be explained below. In the method using coordinate 
conversion of images, upon displaying a pair of display images on the image displays 1, at least one display image is 
subjected to image coordinate conversion processing to rotate it through a predetermined angle relative to an original 
display image, and the rotated image is displayed. 

[0062] Fig. 1 7 is a schematic view showing principal part of the third embodiment of the present invention. The dif- 
ference between this embodiment and the first embodiment is that display images are rotated by electronically attaining 
coordinate conversion of images using image signal modulating portions without tilting the image displays 1 . 
[0063] More specifically, electrical signals supplied from the image signal generating portion 2 to the image displays 
1 are subjected to coordinate conversion processing of images in right and left image signal modulating portions 7 R and 
7 L (7) to rotate the right and left display images 4 R and 4 L> and the rotated images are displayed. 
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[0064] Figs. 18A and 18B are explanatory views of the image signal modulating portion 7 of the third embodiment. 
Fig. 1 8A shows the arrangement of the image signal modulating portion when the image signal supplied from the image 
signal generating portion 2 is an analog video signal. An analog image signal generated by the image signal generating 
portion 2 is converted into a digital signal by an A/D converter 71 , and the digital signal is supplied to a memory circuit 

5 72. On the memory circuit 72, coordinate conversion processing such as movement, interpolation, thin-out processing, 
and the like is performed in units of pixels in accordance with a command signal supplied from a control circuit 73, 
thereby modulating the rotational angle of an image. The control circuit 73 has a knob 74 for adjusting the image rota- 
tional angle, and a user can freely adjust the image rotational angle using this knob. The digital image signal which is 
modulated on the memory circuit 72 is converted into an analog image signal by a D/A converter 75, and the analog 

w image signal is supplied to the corresponding image display 1. Of course, if the image display 1 can receive a digital 
signal, the D/A converter 75 may be omitted. 

[0065] Fig. 18B shows the arrangement of the image signal modulating portion 7 L (7 R ) when an image signal sup- 
plied from the image signal generating portion 2 is a digital signal. A digital image signal generated by the image signal 
generating portion 2 is directly supplied to the memory circuit 72. On the memory circuit 72, coordinate conversion 

15 processing such as movement, interpolation, thin-out processing, and the like is performed in units of pixels in accord- 
ance with a command signal supplied from the control circuit 73, thereby modulating the rotational angle of an image. 
The control circuit 73 has the knob 74 for adjusting the image rotational angle, and a user can freely adjust the image 
rotational angle using this knob. The image display 1 can receive a digital signal, and the modulated digital image signal 
is directly transmitted to the image display 1 . 

20 [0066] As described above, with the above-mentioned arrangements for analog and digital image signals, a desired 
rotational angle can be provided to the display images 4 R and 4 L to be displayed on the right and left image displays 1 R 
and 1 L . In the arrangement of the third embodiment, the right and left optical systems 30 R and 30 L are perfectly sym- 
metrical with each other, and there are no optical factors for rotating the right and left images. Therefore, the rotational 
angles of the display images 4 R and 4 L are directly reflected in those of the corresponding observation images (virtual 

25 images) 4 R and 4 L '. 

[0067] Fig. 1 9 shows the display images 4 R and 4 L for the right and left eyes to be displayed on the right and left 
image displays 1 R and 1 L in this embodiment. At this time, since the display image 4 q for the right eye is rotated clock- 
wise through 1.5°, and the display image 4 L for the left eye is rotated counterclockwise through 1.5°, the observation 
images 4 R ' and 4 L * are similarly displayed in the rotated states. 

30 [0068] As described above, when at least one of the right and left display images is rotated by the above-mentioned 
coordinate conversion processing of an image, the tilt angle of the two observation images can be adjusted by the 
above-mentioned method in desired directions within the range from an angle larger than 0° to an angle equal to or 
smaller than 3°, and preferably, within the range from 1 ° to 3°, thus obtaining two observation images which are tilted in 
a V-shaped pattern. In this manner, good "observability" of a stereoscopic image can be obtained as in the first embod- 

35 iment. 

[0069] In this embodiment as well, since the two observation images need only be tilted relative to each other, the 
image signal modulating portion 7 may be arranged for only one image, and only one input image may be tilted from 
the horizontal state. 

[0070] Method 2) using optical systems will be described below. In the method using optical systems, when a ster- 
eo eoscopic image display apparatus has a pair of optical systems for forming right and left observation images at prede- 
termined positions, an image is rotated during the formation process of a virtual image (observation image) from a 
display image by each optical system. When this method is adopted, the display images 4 R and 4 L for the right and left 
eyes are displayed on the right and left image displays 1 so as not to be tilted relative to each other, and the observation 
images are rotated somewhere in the right and left optical systems. 
45 [0071] Fig. 20 is a schematic view showing principal part of the fourth embodiment of the present invention. In this 
embodiment, the observation images are rotated by image rotators. This embodiment is substantially the same as the 
first embodiment except that the right and left image displays 1 R and 1 L are not tilted relative to each other, and image 
rotators are added to the optical systems. In Fig. 20, image rotators 8 R and 8 L are used for tilting the observation 
images. 

so [0072] Figs. 21 A and 21 B are perspective views of the image rotators. The image rotator is an optical element 
which rotates an image transmitted therethrough through 26° when it is rotated through 6° about its optical axis. For 
example, an image rotator using three mirrors, as shown in Fig. 21 A, and an image rotator using a prism, as shown in 
Fig. 21 B, are available. In this embodiment, the image rotator of the type shown in Fig. 21 B is used. The image rotators 
8 R and 8 L can be rotated through an arbitrary angle about their optical axes using support members 8a and knobs 8b. 

55 Note that a structure for electrically rotating the image rotators 8 R and 8 L may be adopted. According to the arrange- 
ment shown in Fig. 20, the rotational angles of the right and left observation images can be adjusted to arbitrary angles. 
[0073] The insertion positions of the image rotators 8 R and 8 L are not particularly limited as long as they are 
inserted in the optical paths. For example, when the image rotators 8 R and 8 L are disposed at positions corresponding 
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to the beam waists of light beams which contribute to formation of virtual images, their sizes can be reduced. On the 
other hand, as shown in Fig. 20, when each image rotator is inserted between the mirror 5 and the pupil (eye) 6 of the 
observer, the rotation direction of the image rotator can be matched with that of the observation image (virtual image). 
[0074J In this embodiment, since the right image rotator 8 R is rotated clockwise (viewed from the exit side of a light 
beam) through 0.5°, and the left image rotator 8 L is rotated counterclockwise (viewed from the exit side of a light beam) 
through 0.5°, the right observation image 4 R ' is rotated clockwise through 1°, and the left observation image 4,' is 
rotated counterclockwise through 1°. Therefore, the two observation images 4 R ' and 4,' are tilted through 2° relative to 
each other. 

[0075] As described above, when the movable image rotator is inserted in at least one of the right and left optical 
systems, the tilt angles of two observation images can be arbitrarily set, so that the two images are tilted in a V-shaped 
pattern relative to each other within the range from an angle larger than 0° to an angle equal to or smaller than 3° and 
preferably, within the range from 1° to 3°. In this manner, good "observability" of a stereoscopic image can be obtained 
as in the first embodiment. 

[0076] In this embodiment' as well, since it suffices to tilt the two observation images relative to each other the 
image rotator 8 may be inserted in only one optical path, and only one observation image may be tilted from the hori- 
zontal state. 

[0077] Even when no optical systems are used in displaying images, the image rotator can be arranged in front of 
one eye of the observer upon observation, thus tilting the observation image on the corresponding side. 
[0078] Note that practicing of the present invention by rotating at least one of the right and left image displays 1 like 
in the first embodiment is regarded as a kind of method 2) using optical systems. 

[0079] Mote that method 1) using coordinate conversion of images and method 2) using optical systems may be 
combined. This example will be described below. 

[0080] Fig. 22 is a schematic view showing principal part of the fifth embodiment of the present invention In this 
embodiment, rotation of the observation images 4 R ' and 4 L ' is realized using both methods 1) and 2) In Fig 22 image 
signal modulating portions 7 R and 7 L rotate the display images by the coordinate conversion processing of images 
Image rotators 8 R and 8 L can also rotate observation images. 

[0081] As described above, in the first to fifth embodiments, in order to display the observation images for the right 
and left eyes while tilting them in a V-shaped pattern relative to each other within the range from an angle larger than 
0° to an angle equal to or smaller than 3°, both the observation images for the right and left eyes are tilted or are rotated 
30 independently. 

[0082] In the first to fifth embodiments, the present invention is applied to an output portion (HMD display appara- 
tus) of the image display apparatus. 

Alternatively, when the present invention is applied to an input portion (input means) of a stereoscopic image display 
apparatus, i.e.. an image recording apparatus, the same effect can be obtained. Embodiments of four different types of 
35 input means will be described hereinafter. 

•Double-Lens Camera Input Means 

[0083] Fig. 23 is an explanatory view of the sixth embodiment of the present .invention. In this embodiment, in order 
40 to set a V-shaped layout of right and left input images upon input, in an image recording apparatus having double-lens 
camera input means, the "method using optical systems" used in the fourth embodiment, i.e., an image rotator is 
applied to one optical system (optical means) of a double-lens camera system. 

[0084] In Fig. 23, an image of an object X is taken by right and left cameras Y R and Y L constituting the double-lens 
camera system. Optical systems 13 R and 13 L form an image of the object X on image recording means 9 R and 9 L to 
obtain mput images X R and X L . A display control portion 14 receives signals of the input images from the double-lens 
camera system, and appropriately displays the received images on image display means 15 as display imaqes by con- 
trolling the received images. 

[0085] In this arrangement, an image rotator 8 is inserted in the optical path of at least one of the optical systems 
of the right and left cameras Y R and Y L . When the image rotator 8 is rotated about its optical axis, the input image X R 
obta.ned on the image recording means 9 R is rotated through an arbitrary angle within the range from an angle larger 
than 0° to an angle equal to or smaller than 3°, and preferably, within the range from 1° to 3°, thereby obtaining the two 
input images X R and X L which are tilted in a V-shaped pattern relative to each other. 

[0086] Fig. 23 shows an example wherein the image rotator 8 is inserted in the optical system of only the right cam- 
era. Furthermore, Fig. 24 shows an example wherein image rotators 8 R and 8( are inserted in the optical paths of both 
the right and left image pick-up optical systems. In this case, the two input images X R and X L can be rotated through 
arbitrary angles in correspondence with the display condition. 

[0087] On the other hand, the lens portions of the cameras remain unmodified, and at least one of the right and left 
image recording means 9 R and 9 L may be rotated in its recording plane within the range from an angle larger than 0° 
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to angle equal to or smaller than 3°, and preferably, within the range from 1° to 3°, so that the central lines, in the vertical 
directions, of the image recording means form a V-shaped layout, thus obtaining the same effect as described above. 
[0088] As described above, in the image recording apparatus with the double-lens camera input means, when the 
image rotator 8 is used in a portion of the double-lens camera system, two input images with a parallax can be input 
5 while being tilted relative to each other. When stereoscopic viewing is performed in a normal image display apparatus 
using these two input images, good "observability" can be provided. 

•Multi-Lens Camera Input Means 

io [0089] In an image recording apparatus having means for inputting input images with a parallax using a multi-lens 
camera system, when the central lines, in the vertical direction, of the input images are set to form a V-shaped pattern, 
an optical system (optical means) for taking an input image for the right eye and an optical system (optical means) for 
taking an input image for the left eye can be optically rotated by the above-mentioned method as in the double-lens 
camera system. 

15 [0090] However, in this case, upon inputting input images with a parallax, the position information of the two eyes 
of an observer with respect to the image display means is required. If the positions of the two eyes of the observer are 
determined with respect to the image display means, two cameras for presenting a stereoscopic image corresponding 
to the positions of the two eyes are selected, and two input images obtained by the selected cameras are tilted to form 
a V-shaped pattern. 

20 [0091] If the positions of the two eyes of the observer with respect to the image display means are unknown, detec- 
tors are added to detect the positions of the two eyes of the observer, two cameras for presenting a stereoscopic image 
corresponding to the detected positions are selected, and two input images obtained by the selected cameras are tilted 
to form a V-shaped pattern. 

[0092] Fig. 25 is an explanatory view of the six-lens camera input means. In Fig. 25, an image of an object X is 
25 taken using a six-lens camera system including cameras Ya to Yf. Optical systems 13a to 13f form the image of the 
object X on image recording means 9a to 9f to obtain input images Xa to Xf with a parallax. Of these input images, only 
one input image corresponds to a display image to be observed by the right eye of the observer, and only one input 
image corresponds to a display image to be observed by his or her left eye. 

[0093] Fig. 26 is an explanatory view of the method for observing display images displayed on the basis of the six 

30 input images with a parallax so as to have a correct parallax in a stereoscopic image display apparatus utilizing a bin- 
ocular parallax. Image display means 10 allows observers to independently observe the six display images with a par- 
allax from the same direction as that in the image pick-up operation. For example, the image display means 10 adopts 
the lenticular method, large convex lens/large concave mirror method, parallax barrier method, or the like. Display 
images 1 1 j are displayed on the image display means 1 0. 

35 [0094] Assume that, as shown in Fig. 26, the position of an observer A is determined so that his or her eyes are 
located on direction axes a and b, and the position of an observer B is determined so that his or her eyes are located 
on direction axes e and f. In this case, when the input images Xa and Xb are displayed as display images for the right 
and left eyes of the observer A, and input images Xe and Xf are displayed as display images for the right and left eyes 
of the observer B, the observers can respectively observe stereoscopic images. 

40 [0095] At this time, the images are optically rotated in the optical paths of the optical systems 13 of the cameras Ya 
and Yb, and Ye and Yf, so that the right and left images recorded on the image recording means 9 are tilted in a V- 
shaped pattern within the range from an angle larger than 0° to an angle equal to or smaller than 3°, and preferably, 
within the range from 1° to 3°, thereby achieving the object of the present invention. An embodiment for realizing this 
operation will be described below. 

45 [0096] Fig. 27 is a schematic view showing principal part of the seventh embodiment of the present invention. In 
this embodiment, the present invention is applied to an image recording apparatus having multi-lens camera input 
means 111. The method of obtaining input images which have a parallax and are tilted relative to each other in this 
embodiment is the same as the input method using the double-lens camera system shown in Fig. 24. Referring to Fig. 
27, the apparatus includes cameras Ya to Yf for taking an image of an object X, image recording means 9a to 9f, optical 

so systems 13a to 13f, and image rotators 8. 

[0097] As shown in Fig. 27, the image rotators 8 are inserted in the optical paths of either ones (the optical systems 
13a and 13f in Fig. 27) of the optical systems of the cameras Ya and Yb, and Ye and Yf, and each image rotator 8 is 
rotated about its optical axis as in the fourth embodiment, thereby rotating the input image Xa or Xf obtained on the 
image recording means 9a or 9f through an arbitrary angle within the range from an angle larger than 0° to an angle 

55 equal to or smaller than 3°, and preferably, within the range from 1 ° to 3°. As a result, the two images Xa and Xb or Xe 
and Xf, which are tilted in a V-shaped pattern relative to each other, can be obtained. (A description of the operation at 
the cameras Yc and Yd will be omitted.) 

[0098] Alternatively, as in the case of the double-lens camera system, the same effect as described above can be 
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obtained by rotating the image recording means 9a to 9f. 
• [0099J Furthermore, as shown in Fig. 28, the image rotators may be inserted in the optical paths of the optical sys- 
tems of all the cameras so as to rotate the input images in correspondence with the display condition 
[0100] When the positional relationship between the two eyes of the observer and the image display means is 
unknown, the seventh embodiment cannot be used. In this case, the following embodiment is used. 
[0101] Fig. 29 is a schematic view showing principal part of the eighth embodiment of the present invention In this 
embodiment, the present invention is applied to a case wherein the positions of the two eyes of an observer with respect 
to image display means 1 0 are unknown in an image recording apparatus using multi-lens camera input means and an 
image display apparatus using the image recording apparatus. In Fig. 29, multi-tens camera input means 111 is the 
same as that in the seventh embodiment. An image input portion 112 inputs a plurality of display images 1 1- to the 
image display means 10. A detector 12 detects the positions of the two eyes of the observer. In this embodiment the 
detector 12 outputs position information of the two eyes of the observer relative to the image display means 10 to the 
.mage input portion 1 12 in real time, and the image input portion 1 12 selects a pair of input images corresponding to 
the two eyes of the observer on the basis of the input information. Thereafter, the input images are rotated relative to 
each other by the same operation as in the seventh embodiment, so that the central lines, in the vertical direction of 
the two mput .mages form a V-shaped pattern. Then, the rotated images are displayed on the image display means 1 0 
In this case, image rotators are preferably rotated in accordance with an electrical signal 

[0102] In the seventh and eighth embodiments, the display images may be rotated using the coordinate conversion 
processing of images without using any image rotators. 

•Input Means Using Synthesis • Processing of Image 

[0103] Input means for generating input images with a parallax by synthesis • processing of images allows genera- 
te of input images from many virtual view points using a smaller number of cameras. The input method of input 
images with a parallax by the synthesis • processing of images is basically the same image input technique as the 
above-mentioned input method of input images using the multi-lens camera system. In this method since all the pieces 
of image .nformation are converted into electrical signals, and images from virtual view points are synthesized • proc- 
essed by arithmetic operations on a computer, rotation of right and left input images is attained by the method usinq 
coordinate conversion of images. 

[0104] Fig. 30 is an explanatory view of generation of an input image from a virtual view point. Input images are 
generated in the following procedure. Input images Xa and Xb with a parallax are input to a computer using a double- 
lens camera system (cameras Ya and Yb). Then, an input image Xc from a virtual view point c is generated by synthe- 
s.s • processing of images attained by arithmetic operations of the computer using image information of these two input 
images Xa and Xb. r 

[0105] Fig. 31 is a schematic view showing principal part of the ninth embodiment of the present invention In this 
embodiment, the present mvention is applied to an apparatus which generates, using two parallax images Xa and Xb 
fetched by two cameras Ya and Yb, an input image from a virtual view point C between the two images, and inputs these 
images to display a stereoscopic image. In Fig. 31 , image display means 1 1 1 displays a plurality of display images 1 1 
w.th a parallax for an observer. An image generating input portion 117 fetches input images Xa and Xb from double- 
lens camera input means 1 1 8, appropriately generates an input image Xc from a virtual view point c in correspondence 
with the observation position of the observer, and inputs the generated image to the image display means 1 1 0 thus dis- 
playing images. 

[0106] The operation of this embodiment will be described below. In the layout shown in Fig 31 it is detected in 
advance that the two eyes, 6 R and 6 L , of the observer are present on direction axes a and c. In this case, the image 
generating input portion 1 1 7 generates the input image Xc from the virtual view point c on the direction axis c using the 
input images Xa and Xb with a parallax, and inputs the generated image to the image display means 1 10 The image 
display means 1 10 displays the input image Xa as a display image Xa' for the right eye of the observer, and displays a 
display image Xc' for the left eye based on the input image Xc. At this time, simultaneously with generation of the input 
image Xc for the left eye, the generated image is rotated counterclockwise by the coordinate conversion processing of 
a computer within the range from an angle larger than 0° to an angle equal to or smaller than 3°, and preferably, within 
the range from 1 • to 3°, and the rotated image is displayed as the display image XC, thereby achieving the object of the 
present invention, i.e., tilting the right and left display images Xa' and Xc' relative to each other in a V-shaped pattern 
[0107] When input images with a parallax are input by the synthesis • processing of images, if the positional rela- 
.onsh.p between the two eyes of the observer and the image display means is known, two input images corresponding 
to the two eyes of the observer are rotated by the coordinate conversion of images, and are displayed as display 
images, thus practicing the present invention. ' 

[0108] On the other hand, when the positional relationship between the two eyes of the observer and the image dis- 
play means is unknown, a detector is arranged as in the eighth embodiment to detect the positions of the two eyes of 
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the observer in real time, and at least one of two input images corresponding to the two eyes of the observer is sequen- 
tially rotated by the coordinate conversion of images of a computer in correspondence with the positional relationship 
relative to the image display means 1 1 0, thus practicing the present invention. 

5 »CG Input Means 

[0109] As described above, the CG technique can generate input images with a parallax from arbitrary view points 
from one three-dimensional information. In this case as well, the present invention can be practiced when image gen- 
eration is performed while executing the coordinate conversion for rotating at least one of input images, to be presented 
io to the right and left eyes of the observer, of a plurality of input images. 

[0110] For example, the CG technique can obtain images corresponding to input images Xa to Xf with a parallax 
obtained by taking an image of a certain object X from six directions a tot using the mu!ti-lens camera system, as shown 
in Fig. 25. 

[0111] When the positional relationship between the two eyes of the observer and the image display means is 
is known, at least one of two input images corresponding to the two eyes of the observer is rotated by the coordinate con- 
version, and is displayed as a display image, thus practicing the present invention. 

[01 1 2] On the other hand, when the positional relationship between the two eyes of the observer and the image dis- 
play means is unknown, a detector is arranged as in the eighth embodiment to detect the positions of the two eyes of 
the observer in real time, and at least one of two input images corresponding to the two eyes of the observer is sequen- 
ce tially rotated by the coordinate conversion of images of a computer in correspondence with the positional relationship 
relative to the image display means 110, thus practicing the present invention. 

[0113] As described above, when the input images with a parallax are input by the CG method, one or both of the 
display images for the right and left eyes can also be tilted relative to each other in a V-shaped pattern within the range 
from an angle larger than 0° to an angle equal to or smaller than 3°, and preferably, within the range from 1 ° to 3°. How- 

25 ever, since subjective evaluation values for the rotational amount of an observation image include personal differences, 
the relative tilt angle between display images is preferably set to be about 2° from the graphs shown in Figs. 1 1 A to 1 23 
so as to prevent a decrease in evaluation value of "observability" of a stereoscopic image to be observed. 
[0114] In the above description, the embodiments in which the present invention is applied to the output and input 
portions of a stereoscopic viewing apparatus have been exemplified. Upon application of the present invention to the 

30 respective stereoscopic image display methods, where and how the present invention is practiced will be explained 
hereinafter. 

[0115] An embodiment in which the present invention is applied to a stereoscopic image display apparatus of the 
anaglyph method will be described below. 

[0116] Fig. 32 is an explanatory view of the general procedure of stereoscopic display of the anaglyph method. Fig. 
35 32 illustrates an object X, color filters 1 81 R and 181 L for the right and left eyes, and eyes 6 R and 6 L of an observer. The 
procedure will be described below. 

(32-a) Two input images with a parallax are obtained by the double-lens camera method, image synthesis • 
processing method, CG method, or the like. 
40 (32-b) Two input images are synthesized to one image. 

(32-c) The synthesized image is displayed in two colors. 

(32-d) Observation images are observed via the color filters 1 81 R and 1 81 L for the right and left eyes. 

[011 7] In order to rotate one of the observation images for the right and left eyes to obtain a V-shaped layout of the 
45 central lines, in the vertical direction, of the two images, at least one of the following methods cart be used. 

(1) The method using the image rotator or the method using electronic coordinate conversion of an image is used 
in step (32-a). 

(2) The method using electronic coordinate conversion of an image is used in step (32-b). 

50 (3) The method using an optical system in which an image rotator is arranged before or after the color filter is used 

in step (32-d). 

[01 1 8] On the other hand, when two input images with a parallax are projected onto a single screen 1 92 using two 
projectors 191 R and 191 L with color filters, as shown in Fig. 33, since the input images need not be synthesized to one 
55 image, the procedure shown in Rg. 34 can be used This procedure will be described below. 

(34-a) Two input images with a parallax are obtained by the double-lens camera method, image synthesis • 
processing method, CG method, or the like. 
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(34-b) Two images are projected via the color filters using the two projectors 1 91 R and 1 91 L on the basis of the two 
input images, and are superimpose-displayed as display images on the screen 1 92. 
(34-c) Observation images are observed via the color filters 181 R and 1 81 L for the right and left eyes. 

[0119] In this case, in addition to the above-mentioned methods, the projection optical system of one of two projec- 
tors may be tilted upon projection of images, so as to tilt the two display images relative to each other in a V-shaped 
pattern. 

[0120] An embodiment in which the present invention is applied to a stereoscopic image display apparatus of the 
polarizing glass method will be described below. 

[0121] Fig. 35 is an explanatory view of the general procedure of stereoscopic image display by the polarizing glass 
method. Fig. 35 illustrates polarizing glasses 21 1 . The procedure will be described below. 

(35-a) Two input images with a parallax are obtained by the double-lens camera method, image synthesis ■ 
processing method, CG method, or the like. 

(35-b) The two input images are displayed as display images on two image display means. 

(35-c) Polarizing plates are arranged on the respective image display means, and the two display images are 
superimpose-displayed using a half mirror. 

(35-d) Observation images are observed via the polarizing glasses. 

[0122] In order to rotate one of the observation images for the right and left eyes to obtain a V-shaped layout of the 
two images, at least one of the following methods can be used. 

(1) The method using an image rotator or the method using electronic coordinate conversion of an image is used 
in step (35-a). 

(2) The method using electronic coordinate conversion of an image is used in step (35-b). 

(3) The method using an image rotator is used in right and left display optical systems in step (35-c). (This method 
cannot in some cases be realized in terms of size.) 

(4) The method using an optical system in which an image rotator is arranged before or after the polarizing glasses 
is used in step (35-d). 

[0123] An embodiment in which the present invention is applied to a stereoscopic image display apparatus of the 
time sharing shutter method will be described below. Fig. 36 is an explanatory view of the general procedure of stere- 
oscopic image display by the time sharing shutter method. Fig. 36 illustrates time sharing shutter glasses 221 . The pro- 
cedure will be described below. 

(36-a) Two input images with a parallax are obtained by the double-lens camera method, image synthesis • 
processing method, CG method, or the like. 

(36-b) The two input images are displayed as display images on image display means in a time sharir manner. 
(36-c) Observation images are observed via the time sharing shutter glasses. 

[0124] In order to rotate one of the observation images for the right and left eyes to obtain a V-shaped layou of the 
two images, at least one of the following methods can be used. 

(1) The method using an image rotator or the method using electronic coordinate conversion of an image is used 
45 in step (36-a). 

(2) The method using electronic coordinate conversion of an image is used in step (36-b). 

(3) The method using an optical system in which an image rotator is arranged before or after the time sharing shut- 
ter glasses 221 is used in step (36-c). 
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[0125] An embodiment in which the present invention is applied to a stereoscopic image display apparatus of the 
large convex lens/large concave mirror method will be described below. Fig. 37 is. an explanatory view of the general 
procedure of stereoscopic image display by the large convex lens/large concave mirror method. Fig. 37 illustrates a 
large convex lens or large concave mirror 231. The procedure will be described below. 

(37-a) n input images with a parallax are input by the multi-lens camera method, image synthesis • processinq 
method, CG method, or the like. 

(37-b) The n input images are simultaneously projected as display images at positions near the large convex lens 
or large concave mirror using n projectors. 
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(37-c) Two out of the plurality of display images projected at the positions near the large convex lens or large con- 
cave mirror are observed as observation images. 

[0126] In this case, in order to rotate one of the observation images for the right and left eyes, two input images 
5 must be specified. If the positions of the two eyes of the observer are determined with respect to the large convex lens 
or large concave mirror, two inputs images which can present a stereoscopic image with respect to the positions of the 
two eyes are selected, and the V-shaped layout of two input images is obtained by at least one of the following methods. 
[0127] If the positions of the two eyes of the observer with respect to the large convex lens or large concave mirror 
are unknown, the positions of the two eyes of the observer are detected using a detector like in the eighth embodiment, 
10 two input images corresponding to the detected positions are selected, and the V-shaped layout of two display images 
corresponding to a stereoscopic image is obtained by at least one of the following methods. 

(1) The method using an image rotator or the method using electronic coordinate conversion of an image is used 
in step (37-a). 

15 (2) The display images are rotated using optical systems in step (37-b). Alternatively, the method using coordinate 

conversion of images is used in a process of displaying images on the projectors. 

[0128] When the method using an optical system in which an image rotator is arranged in front of one eye of the 
observer is used in step (37-c), the V-shaped layout of two observation images corresponding to a stereoscopic image 
20 can be obtained independently of the positions of the two eyes of the observer. 

[0129] An embodiment in which the present invention is applied to a stereoscopic image display apparatus of the 
lenticular method will be described below. Fig. 38 is an explanatory view of the general procedure of stereoscopic 
image display by the lenticular method. Fig. 38 illustrates a lenticular lens 241 . A display device 242 comprises a liquid 
crystal panel or CRT The procedure will be described below. 

25 

(38-a) n input images with a parallax are input by the multi-lens camera method, image synthesis • processing 
method, CG method, or the like. 

(38-b) The width of each input image is compressed to 1/n, the compressed image is horizontally divided into n por- 
tions, and divided pixels are picked up and arranged one by one from the n input images to be synthesized into one 
30 image. 

(38-c) The synthesized image is displayed on the display surface of the display device. 
(38-d) The displayed image is observed via the lenticular lens. 

[0130] In order to tilt observation images observed by the two eyes of the observer relative to each other in this 
35 method, position information of the two eyes of the observer is required. 

[0131] If the positions of the two eyes of the observer are determined with respect to the display device, two input 
images which can present a stereoscopic image with respect to the positions of the two eyes are selected, and the two 
input images are tilted in a V-shaped pattern by at least one of the following methods. 

[0132] If the positions of the two eyes of the observer with respect to the display device are unknown, the positions 
40 of the two eyes of the observer are detected using a detector as in the eighth embodiment, two input images corre- 
sponding to the detected positions are selected, and the two input images are tilted in a V-shaped pattern by at least 
one of the following methods. 

(1) The method using electronic coordinate conversion of images is performed in step (38-a). 
45 (2) The method using electronic coordinate conversion of images is performed in step (38-b). 

[0133] Alternatively, observation images may be rotated using the optical method by arranging an image rotator in 
front of one eye of the observer in step (38-d) without processing the input images. With this method, the position infor- 
mation of the two eyes of the observer is not required. 
50 [0134] With the above-mentioned procedure, even in the stereoscopic image display by the lenticular method, the 
two observation images to be observed can always be tilted relative to each other in correspondence with the position 
of the observer within the range from an angle larger than 0° to an angle equal to or smaller than 3°, and preferably, 
within the range from 1° to 3°, thus obtaining good "observability". 

[0135] Fig. 39 is an explanatory view of stereoscopic image display by the lenticular method by means of image 
55 projection. In this method, n parallax images are projected onto a transmission type double lenticular screen 252 using 
n projectors 251, and an observer observes images via the double lenticular screen 252. Fig. 40 shows the procedure 
in this case. The procedure will be described below. 
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(A °2 ] !! 5* ima9eS With 3 Para " ax are input by the mul «-' e ns camera method, image synthesis - processing 
method, CG method, or the like. K a 

(40-b) The n input images are projected onto the double lenticular screen 252 using the n projectors 
(40-c) The displayed images are observed via the double lenticular screen 252. 

[0136] In order to tilt observation images observed by the two eyes of the observer relative to each other in this 
method, position information of the two eyes of the observer is required. 

[0137] If the positions of the two eyes of the observer are determined with respect to the double lenticular screen 
252, two input .mages which can present a stereoscopic image with respect to the positions of the two eyes are 
selected, and .mages on the projectors which project the two input images are tilted in a V-shaped pattern by at least 
one of the following methods. H y 

[01 38] If the positions of the two eyes of the observer with respect to the double lenticular screen 252 are unknown 
the posrt.ons of the two eyes of the observer are detected using a detector like in the eighth embodiment two input 
.mages corresponding to the detected positions are selected, and images on the projectors which project the two input 
.mages are tilted in a V-shaped pattern by at least one of the following methods. 

(1) The method using electronic coordinate conversion of images is used in step (40-a) 

(2) The method using an image rotator in a projector optical system or the method using electronic coordinate con- 
version of images is used in step (40-b). 

[0139] Alternatively, observation images may be rotated using the optical method by arranging an image rotator in 
front of one eye of the observer in step (40-d)without processing the input images. With this method, the position infor- 
mation of the two eyes of the observer is not required. 

[0140] With the above-mentioned procedure, even in the stereoscopic image display by the lenticular method bv 
means of image projection, the two observation images to be observed can always be tilted relative to each other in 
correspondence with the position of the observer within the range from an angle larger than 0° to an angle equal to or 
smaller than 3°, and preferably, within the range from 1 0 to 3°, thus obtaining good "observability" 
[0141] An embodiment in which the present invention is applied to a stereoscopic image display apparatus of the 
parallax barr.er method will be described below. Fig. 41 is an explanatory view of stereoscopic image display by the par- 
allax barrier method. Fig. 41 illustrates a parallax barrier 271 . This method is substantially the same as the stereoscopic 
.mage display by the lenticular method. In this case as well, a liquid crystal panel or CRT must be used as a display 
device. The procedure will be described below. 

(4 V a) w ima9eS With 3 para " aX are input by the multi - |en « camera method, image synthesis • processing 
35 method, CG method, or the like. a 

(41 -b) The width of each input image is compressed to 1/n, the compressed image is horizontally divided into n por- 
tions, and div.ded pixels are picked up and arranged one by one from then input images to be synthesized into one 
image. 

(41 -c) The synthesized image is displayed on the display surface of the display device. 
(41 -d) The displayed image is observed via the parallax barrier. 

[0142] In order to tilt the observation images observed by the two eyes of the observer relative to each other in this 
method, position information of the two eyes of the observer is required. 

[0143] If the positions of the two eyes of the observer are determined with respect to the display device two input 
as .mages which can present a stereoscopic image with respect to the positions of the two eyes are selected and the two 
input images are tilted in a V-shaped pattern by at least one of the following methods 

[0144] If the positions of the two eyes of the observer with respect to the display device are unknown, the positions 
of the two eyes of the observer are detected using a detector as in the eighth embodiment, two input images corre- 
sponding to the detected positions are selected, and the two input images are tilted in a V-shaped pattern by at least 
so one of the following methods. 

(1) The method using electronic coordinate conversion of images is performed in step (41 -a). 

(2) The method using electronic coordinate conversion of images is performed in step (41 -b). 

[0145] Alternatively, observation images may be rotated using the optical method by arranging an image rotator in 
front of one eye of the observer in step (41 -d) without processing the input images. With this method, the position infor- 
mation of the two eyes of the observer is not required. 

[0146] With the above-mentioned procedure, even in the stereoscopic image display by the parallax barrier 
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method, the two observation images to be observed can always be tilted relative to each other in correspondence with 
the position of the observer within the range from an angle larger than 0 C to an angle equal to or smaller than 3°, and 
preferably, within the range from 1° to 3°, thus obtaining good "observability". 

[0147] The conventional stereoscopic image display utilizing a binocular parallax has no established display condi- 
5 tion between the observation images for the right and left eyes, but is premised on the simple assumption that images 
for the right and left eyes are displayed under an identical condition. No image display apparatus which sets the display 
condition for images for the right and left eyes in consideration of effects to be given to an observer is available. 
[0148] Similarly, as for the image input apparatus for inputting images to the image display apparatus for displaying 
a stereoscopic image, no apparatus which sets the display condition for images for the right and left eyes in considera- 
w tion of effects to be given to an observer is available. 

[0149] However, as a result of examination of effects of different display conditions for images for the right and left 
images with respect to an observer in the stereoscopic image display utilizing a binocular parallax, the present inventors 
found that n the observability of a stereoscopic image could be improved when a magnification difference was set 
between images for the right and images". 
15 [0150] Note that the magnification difference is defined as: 



Magnification Difference = 
Size of image for right eye to be observed by observer - 
Size of image for left eye to be observed by observer „ 1QQ . D/ . 
Size of image for left eye to be observed by observer 



[0151] Figs. 42A and 42B are graphs showing the examination results. In each graph, the magnification difference 
(%) is plotted along the abscissa, and the evaluation value obtained by distributed processing of the subjective evalua- 

25 tion values of "observability" of a stereoscopic image for a plurality of observers is plotted along the ordinate. 

[0152] Fig. 42A shows a case wherein the magnification difference is positive, and Fig. 42B shows a case wherein 
the magnification difference is negative. As can be seen from Figs. 42A and 42B, the evaluation value of "observability" 
of a stereoscopic image obtained upon generation of a magnification difference of about ±2% is higher than that 
obtained in a case wherein no magnification difference is generated between the right and left images (magnification 

30 difference = 0%), and the evaluation value is high within the range of 0 < Imagnification difference! < 3 (%). 

[0153] As can be seen from the above description, a stereoscopic image can be made easy to observe when a 
magnification difference is given between the images for the right and left eyes to be observed by the observer. 
[0154] Fig. 43 is a schematic view showing principal part of the 1 0th embodiment of the present invention. In this 
embodiment, the present invention is applied to an image display apparatus for independently presenting a plurality of 

35 images with a parallax to the two eyes of an observer via optical systems and allowing the observer to recognize a ster- 
eoscopic image, i.e., to a so-called HMD (head-mounted display). 

[0155] Referring to Fig. 43, image displays (image display means) 1 R and 1 L (subscripts R and L respectively indi- 
cate elements for the right and left eyes; the same applies to elements to be described below) comprise liquid crystal 
display elements (to be abbreviated as LCDs hereinafter) with backlights in this embodiment. As the image display, a 
40 CRT, a plasma display, or the like may be used. 

[0156] An image signal generating portion 2 generates image signals of display images 4 R and 4 L to be respec- 
tively presented to the right and left eyes of an observer, and transmits the image signals of the display images 4 R and 
4 L to the image displays 1 R and 1 L via a magnification modulator 7. 

[0157] Lenses 3 R * and 3 L have a role of forming, in an enlarged scale, virtual images (observation images) 4 R and 
45 4 L ' of the display images 4 R and 4 L displayed on the display portions of the image displays 1 R and 1 L at positions in 
front of the observer. 

[0158] Mirrors 5 R and 5 L bend the optical axes of the lenses 3 R and 3 L . On the optical axes bent by the mirrors 5 R 
and 5 L , eyes 6 R and 6 L of the observer who wears the apparatus of this embodiment are located, and the observer can 
observe the observation images 4 R ' and V enlarged in the optical axis (forward) directions. 
so [0159] In this embodiment, since the mirrors 5 n and 5 L are used, the entire apparatus can be made compact, thus 
facilitating mounting of this apparatus on the head portion of the observer. When the mirrors 5 themselves have an opti- 
cal power, a further size reduction and improvement of optical performance can be attained. 

[0160] When half mirrors are used as the mirrors 5 R and 5 L , a background image can be superposed on the obser- 
vation images 4 R and 4 L \ 

55 [0161] In the above-mentioned arrangement when images with a parallax are input from the image signal generat- 
ing portion 2 to the image displays 1 R and 1 L , the display images 4 R and 4 L displayed on the image displays 1 R and 1 L 
are presented as observation images 4 R and 4 L * to the right and left eyes of the observer via optical systems 30 R and 
30 L , and the observer can stereoscopically observe the observation images. 
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[0162] In this embodiment, the optical systems 30 R and 30 L have substantially the symmetrical arrangement about 
the vertical direction, except that the lens 3 R ' for the right eye can vary its focal length. Since the focal length can be 
varied, the size of the observation image 4 R ' can be adjusted. 

[0163] In this embodiment, the magnification modulator 7 modulates the size of the display image 4 R from the 
image signal generating portion 2 by coordinate conversion processing. 

[0164] Fig. 44 is a schematic view showing principal part of a modification of this embodiment. As compared to the 
arrangement shown in Fig. 43, the mirrors 5 R and 5 L are omitted. 

[0165] in this modification, the eyes 6 R and 6 L of the observer are located on the optical axes of optical systems 

m«« ^ 0bSefVer ° an ° bSerVe thS observation ima 9 es V and 4 L ' enlarged in the optical axis directions 

[0166] As in the above-mentioned embodiment, the optical systems 23 R and 23 L and the positions of the eves 6 D 
and 6 L of the observer are set at substantially the symmetrical positions about the vertical direction, and the right and 
left optical systems 23 R and 23 L form the observation images 4 R ' and 4 L ' at substantially the same position on the 
space. 

[0167] For this reason, when the images 4 R and 4 L for the right and left eyes including a binocular parallax are 
respectively displayed on the right and left image display, an observer can easily enjoy stereoscopic observation 
[0168] In this modification, the right optical system 23 R can vary its local length, thereby changing the size of the 
observation image 4 R \ The magnification modulator 7 is inserted between the image signal generating portion 2 and 
the right .mage d.splay 1 R , and changes the size of the display image 4 R by coordinate conversion processing 
[0169] In the above-mentioned embodiment and modification, the "observability" is improved by chanqinq the 
images of the right and left eyes to be observed by the observer, i.e. by assigning an appropriate magnification dtffer- 
ence to the observation images 4 R ' and 4 L \ 
[0170] Note that the magnification difference is defined by: 

Magnification Difference = 
Size of image for right eye to be observed by observer - 
Size of image for left eye to be observed by observer « 
Size of image for left eye to be observed by observer 100 (° /o ) 

30 [0171] As a method of assigning the magnification difference, the following two methods are used. 

1 ) Image Coordinate Conversion Method 

2) Asymmetrical Optical System Method 

35 [0172] In image coordinate conversion method 1 ), upon displaying a pair of display images on the image displays 
1 at least one d.splay image is subjected to coordinate conversion processing of an image to generate a magnification 
difference between the display images 4 R and 4 L for the right and left eyes. 

[0173] On the other hand, in asymmetrical optical system method 2), when the stereoscopic image display appa- 
ratus has a pair of optical systems for forming right and left images at predetermined positions, the optical systems are 
40 asymmetrically arranged (i.e., to have different magnification factors), thereby finally generating a magnification differ- 
ence between the right and left images to be displayed. 

[0174] These methods will be described in detail in embodiments to be described below 

[0175] In this embodiment, the "observability is improved by assigning a magnification difference between the 
observation images 4 R and 4 L * as described above. 

[0176] Fig. 45 is a schematic view showing principal part of the 1 1th embodiment of the present invention Fig 46 
shows the arrangement of the magnification modulator 7 when image signals supplied from the image signal generatinq 
port,on 2 are analog video signals, and Fig. 47 shows the arrangement of the magnification modulator 7 when image 
signals supplied from the image signal generating portion 2 are digital video signals. The same reference numerals in 
Figs. 45 to 47 denote the same parts as in Fig. 43, and a detailed description thereof will be omitted 
l V 7 Hu ln thiS embodiment ' °P tical sterns 30 R ' and 30 L for the right and left eyes have a symmetrical arrangement 
about the vertical direction, and form the display images 4 R and 4 L on the image displays 1 R and 1 L at the same mag- 
nification factor, thereby forming the observation images 4 R * and 4 L \ 

[0178] More specifically, in this embodiment, the magnification difference is generated in such a manner that an 
image signal from the image signal generating portion 2 is subjected to coordinate conversion processing in the maq- 
nif.cat.on modulator 7 to change the size of the display image 4 R to be displayed on the image display 1 R 
[0179] The image coordinate conversion method in the magnification modulator 7 will be described in detail below 
with reference to Figs. 46 and 47. 

[0180] Referring to Fig. 46, an analog image signal generated by the image signal generating portion (or generator) 
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2 is converted into a digital signal by an A/D converter 31 , and the digital signal is supplied to a memory circuit 32. The 
size of the image is modulated on the memory circuit 32 by performing coordinate conversion processing such as inter- 
polation, thin-out processing, or the like in units of pixels in accordance with a command signal supplied from a control 
circuit 33. 

5 [0181] The control circuit 33 has a knob 34 for adjusting the degree of magnification modulation, and a user can 
freely adjust the magnification using this knob 34. The digital image signal modulated on the memory circuit 32 is con- 
verted into an analog image signal by a D/A converter 35, and the analog signal is supplied to the corresponding image 
display 1 . Of course, if the image display 1 can receive a digital signal, the D/A converter 35 can be omitted. 
[0182] When an image signal supplied from the image signal generating portion 2 is a digital signal, the magnrfica- 

w tion modulator 7 has the arrangement shown in Fig. 47. 

[0183] An image signal generated by the image signal generating portion 2 is directly supplied to the memory circuit 
32. On the memory circuit 32, the size of the image is modulated by performing coordinate conversion processing such 
as movement, interpolation, thin-out processing, or the like in units of pixels in accordance with a command signal sup- 
plied from a control circuit 33. 

15 [0184] The control circuit 33 has the knob 34 for adjusting the degree of magnification modulation, and a user can 
freely adjust the magnification using this knob 34. 

[0185] The image display 1 can receive a digital signal, and the modulated digital image signal is directly transmit- 
ted to the image display 1 . 

[0186] As described above, with the above-mentioned arrangements for analog and digital image signals, a desired 
20 magnification difference can be assigned to the display images 4 R and 4 L displayed on the right and left image displays 
1 R and 1 L . 

[0187] In this embodiment, since the optical systems 30 R ' and 30 L have a symmetrical arrangement, and are 
designed to have the same imaging magnification, the magnification difference between the display images 4 R and 4j_ 
is directly reflected in that between the observation images 4 R ' and 4 L ". 
25 [0188] Figs. 48 A and 48B are explanatory views showing the display images 4 R and 4 L displayed on the image dis- 
plays 1 R and 1 l upon generation of the magnification difference in this embodiment. 

[0189] Fig. 48A shows a case wherein a magnification difference of +2% is generated between the display images 
4 R and 4 L . 

[0190] At this time, if the size of the display image 4 L is assumed to be 1 00, the size of the display image 4 R is 1 02. 
30 If both the right and left optical systems 30 R and 30 L have an enlargement magnification of x1 0, the observation images 

4 R ' and 4 L * respectively have sizes of 1000 and 1020, and the magnification difference is +2% as in the original images. 

[0191] Fig. 48B shows a case wherein a magnification difference of -2% is generated between the display images 

4 R and \. At this time, if the size of the display image 4 L is assumed to be 1 02, the size of the display image 4 R is 1 00. 

[0192] In this embodiment, as described above, the "observability" of a stereoscopic image is improved by image 
35 processing (image coordinate conversion method) of the display images 4 R and 4 L . 

[0193] Note that the sizes of the observation images 4 R ' and 4 L ' need not always be changed by evenly changing 

the entire frames. For example, the horizontal or vertical size may be changed (Figs. 49A and 49B), or the size may be 

partially changed (Figs. 49C and 49D). 

[0194] Fig. 50 is a schematic view showing principal part of the 12th embodiment of the present Invention. The same 
40 reference numerals in this embodiment denote the same parts as in the 10th embodiment shown in Fig. 43, and a 
detailed description thereof will be omitted. 

[01 95] In this embodiment, although lenses 3 R and 3 L for the right and left eyes have the same arrangement, optical 
systems 30 R ' and 30 L for the right and left eyes have asymmetrical positional relationships between the lenses 3 R and 
3 L , the display images 4 R and 4 L , and the observation images 4 R * and 4 L \ thereby generating a magnification difference 
45 between the observation images 4 R ' and 4 L \ 

[0196] At this time, the positional relationships between the respective elements are determined based on an opti- 
cal paraxial theory, as shown in Fig. 51 . 
[0197] In Fig. 51, 

50 H L , H'(_: the principal points of the lens 3 L 
F L , P L : the focal points of the lens 3 L 
V f L : the focal lengths of the lens 3 L 

z L , z* L : the distances from the focal points F L and F' L to the display images (observation images) 4 L and 4 L * 

55 [0198] Similarly, 

H Rt H' R : the principal points of the lens 3 R 
F R , P R : the focal points of the lens 3 R 
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f R , f' R : the focal lengths of the lens 3 R 

z R , z' R : the distances from the focal points F R and F R to the display images (observation images) 4 R and 4 R ' 

[0199] From the paraxial imaging formula, the magnifications, p R and p L> of the right and left optical systems are 
respectively given by: 

P R = -f R /z R =-z' R /f R (1) 
P L = -f' L /z L = -z* L /f L 

If the sizes of the display images 4 R and 4 L are equal to each other, as described above, the magnification difference 
AP (%), is given by: 

A P = P^x100= , -Bii^iRx 1 00(%) (2) 
P L L R 

[0200] Since f R = f L 

Ap = ^ll5x100(%) (3) 
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[0201] Therefore, the distances between the right and left lenses 3 R and 3 L and the display images 4 R and 4, can 
25 be set to satisfy equation (3) within the range of 0 < lAgl < 3 (%). 

[0202] If the distances z R and z L are set to be different from each other, the distances z' R and z' L also become dif- 
ferent from each other, and hence, the observation images 4 R ' and 4 L ' have different positions. 

[0203] In this case, the imaging positions of the observation images 4 R » and 4 L * may be kept equal to each other 
so that the distances from the eyes 6 R and 6 L of the observer to the lenses 3 R and 3 L have a difference corresponding 
30 to that between the distances z' R and z' L : 



2 , R -2' L = ^R 2 (^z R -1/z L ) 



[0204] At this time, when a magnification difference of 2% is to be generated between the observation images 4 R ' 
and 4 L ', if the enlargement magnification is x1 0, the enlargement magnification of the optical system for the right eye is 
xto.2, and the sizes of the display images 4 R and 4 L are 100, the sizes of the observation images 4 R ' and 4 L ' respec- 
tively become 1 020 and 1 000, thus obtaining a magnification difference of +2%. 

[0205] In this embodiment, as described above, the "observability- is improved by modifying the arrangements of 
the optical systems 30 R * and 30 L for displaying images (by setting different imaging magnifications). 
[0206] Note that a change in size of the observation images 4 R ' and 4 L ' by means of the asymmetrical arrangement 
of the optical systems 30 R and 30 L need not always be attained by evenly changing the entire frame. For example, the 
horizontal or vertical size may be changed (using, e.g., a cylindrical lens) or the size may be partially changed. 
[0207] Fig. 52 is a schematic view showing principal part of the 13th embodiment of the present invention. The 
arrangement of this embodiment is substantially the same as that of the 1 2th embodiment shown in Fig. 50, except that 
45 a magnification correction lens 8 is added to the lens 3 R for the right eye to constitute a lens 33 R . 

[0208] In Fig. 52, the lens 33 R and the mirror 5 R constitute an optical system 30 R M for the right eye. 
[0209] In this embodiment, since the magnification correction lens B is added, the distance from the display image 
4 R to the lens 33 R is changed, thereby generating a magnification difference between images formed by the lenses 33 D 
and 3 L . H 

so [021 0] At this time, the positional relationships between the respective elements are determined based on an opti- 
cal paraxial theory, as shown in Fig. 51 . Note that the lens 3 R in Fig. 51 is replaced by the lens 33 R . 
[0211] First, from the Newton's equation. 



f L 2 = ZL ,Z L' (4) 

f R 2 = z R -z R ' 



However, since the right and left observation images 4 R ' and 4 L ' are formed at the same positions (i.e. z' R = z\ ), we 
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have: 

2 2 

f L /Z L = f R /Z R (5) 

5 [0212] On the other hand, the magnifications, p R and p L , of the lenses 33 R and 3 L are: 

l*R = - f R /z R (6) 

p L = -r L /z L 



10 



35 



If the sizes of the display images 4 R and 4 L are equal to each other, as described above, the magnification difference 
A|5 (%) is given by: 

a P = x too = ^jde x 100 (%> (7) 

Simultaneous solution of equations (5) and (7) for the focal length f R and the distance z R gives: 

f f L 

R 1+O.O1A0 ( 8 ) 

(9) 



Zn = 



25 " (1+0.01 Ap) 2 



[0213] The optical position of the lens 33 R for the right eye is adjusted, so that its focal length f R and distance z R 
satisfy the conditions of equations (8) and (9). 
30 [0214] On the other hand, if the focal length of the lens 33 R is equal to the focal length f L of the lens 3 L , and the 
principal point distance of the lens 33 R is represented by d, the local length f R of the lens 33 R is given by: 



f L -f8 



R f L +f8-d 
From equation (5), we have: 

,8= c^TZ]j < 10 > 



[0215] When the magnification difference Ap is to be obtained, the magnification correction lens 8 that satisfies the 
condition of equation (10) can be used. 
45 [021 6] Like in the 1 0th embodiment shown in Fig. 43, one of the tenses 3 L and 33 R may constitute a so-called zoom 
lens system, so that a user can arbitrarily adjust the magnification difference. 

[021 7] In particular, the degree of improvement in "observability" obtainable by generation of a magnification differ- 
ence largely depends on personal differences (personal favors), and the adjustment range is preferably set to fall within 
the range from about -3% to +3%. 
50 [0218] In this embodiment, the magnification difference is obtained by the asymmetrical arrangement of the optical 
systems 30 R ' and 30 L . Alternatively, as in the embodiment shown in Fig. 43, the magnification modulator 7 may also be 
arranged between the image signal generating portion 2 and the image displays 1 R and 1 L attain the magnification dif- 
ference by both the means. 

[0219] At this time, control means (not shown) may be arranged to recognize the magnification difference gener- 
55 ated by the optical systems 30 R and 30 L , and the magnification difference generated by the magnification modulator 7, 
and may control the optical systems 30 R and 30 L and/or the magnification modulator 7 to obtain a desired magnifica- 
tion difference. 

[0220] Let AP1 (%) be the magnification difference of the display image 4 R with respect to the display image 4 L gen- 
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erated by the image coordinate conversion method, and AJ52 (%) be the magnification difference generated by the 
asymmetrical optical system method. The magnification difference, A, of the observation image 4 R ' with respect to the 
observation image 4 L ' is then given by: K 

5 A = A|}2 + Ap1 + (APTAP2/100) (%) 

Therefore, in this case, A|3t and A|12 are adjusted so that A falls within the range from -3% to +3% 

[0221] In th.s manner, the image coordinate conversion method and the asymmetrical optical system method can 

be combined. ' 

10 [0222] Note that the magnification difference is generated within the range from -3% to +3% However since sub- 
jective evaluation values for the magnification difference include personal differences, the magnification difference is 
most preferably set to be about ±2% from the graphs shown in Figs. 42A and 42B so as to prevent a decrease in eval- 
uation value of "observability" of a stereoscopic image to be observed. 

[0223] As described above, in this embodiment, the magnification difference is adjustable, so that the observer can 
is select the magnification difference at which an image is most observable. 

[0224] Fig. 53 is an explanatory view of the 14th embodiment of an image pick-up apparatus accordinq to the 
present invention. a 

[0225] In this embodiment, the present invention is applied to an image pick-up apparatus for taking parallax 
images us.ng a double-lens camera system. This apparatus can improve the "observability" for an observer who 
20 observes taken parallax image which are input to an image display apparatus, an image recording apparatus or the like 
and ,s displayed on the image display apparatus, or for an observer who observes parallax images which are repro- 
duced from a recording medium recorded by the image recording apparatus and are displayed on the image display 
apparatus. K y 

25 came/a system 9 ' ^ *" ima96 ° f * * ^ ^ cameras Y « and Y l constituting the double-lens 

[0227] The cameras Y R and Y L respectively comprise image pick-up optical systems 60 R and 60, including image 
P ' C h £ !T,! 63 ". and v 63 L- Th - ese ima S e P ick -"P le "ses 63 R and 63 L respectively form the image (taken images 64 R 

»"J fV ? f f ° n ima9e t3king SUrfaCeS ° f image tekin9 means 9 * and 9 L- The ^age taking means 9 R 
and 9 L take the taken images 64 R and 64 L formed on their image taking surfaces, thus obtaining corresponding image 
30 sicjnals. 

[0228] A magnification correction lens 68 is arranged in the optical path of at least one of the right and left imaae 
pick-up optical systems 60 R and 60 L (in the right image pick-up optical system 60 R in Fig. 53) 

[0229] In this embodiment, the magnification difference between the taken images 64 R and 64, is obtained by 
adjusting the focal length of the image pick-up optical system 60 R (i.e., the synthesized focal length of the magnification 
35 correct.on lens 68 and the right lens system 63 R ), and the distance from the image pick-up optical system 60 R to the 
image taking surface of the image taking means 9 R so as to change the size of the taken image 64 R obtained on the 
image taking surface of the means 9 R . 

[0230] Note that the magnification difference is defined by: 

40 _. 

Magnification Difference Ai = s * e <* taken image 64 R -Size of taken image 64 L , 

Size of taken image 64 [ iuo(/o) 

[0231] In this embodiment, the "observability" upon displaying the taken images 64 R and 64, on an image display 
45 apparatus or the like is improved by generating a magnification difference between the taken images 64 R and 64. 

[0232] In particular, the "observability is improved by modifying the arrangement of the image pick-up optical sys- 
tems 60 R and 60 L (by setting different imaging magnifications). 

Sf 331 c- ^f rnative,y - as shown in Fi 9- 54- image pick-up optical system (the right image pick-up optical system 
60 R ,n F.g. 54) may comprise a zoom lens system 68" which can vary its focal length, to allow adjustment of the size of 
so the taken image 64 R , so that the magnification difference Ai is adjustable within the range from -3% to +3% 

[0234] With this arrangement, the effect of improving the "observability" can be adjusted, and the photographer can 
select a most desirable magnification difference Ai. 

[0235] Furthermore, as shown in Fig. 55. both image pick-up optical systems 60 R ' and 60 u - may allow to vary their 
focal lengths, and control means 69 may be arranged, so as to allow a change in image pick-up magnifications while 
55 maintaining a desired magnification difference Ai. 

[0236] In this arrangement, the control means 69 detects the focal lengths (zoom positions) of the image pick-up 
optical systems 60 R ' and 60 L ', and controls to maintain the magnification difference Ai within the range of 0 < Imaonifi- 
cation difference Ail < 3. a 
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[0237] As shown in Fig. 56, a magnification modulator 65 may be arranged, and may perform image processing of 
the taken images 64 R and S\ from the cameras Y R and Y L by the image coordinate conversion method so as to gen- 
erate a magnification difference. With this arrangement, the 'observability" upon displaying the taken images 64 R and 
64 L on, e.g., an image display apparatus, can be improved. 
5 [0238] Fig. 57 is a schematic view showing principal part of the 1 5th embodiment of the present invention. In this 
embodiment, the present invention is applied to an image pick-up apparatus for taking parallax images using a multi- 
lens camera system. This apparatus takes parallax images, so that a magnification difference is generated between 
those for the right and left eyes of the parallax images. 

[0239] In Fig. 57, an image of an object X is taken by cameras Ya to Ye. The image of the object X is formed on the 
io image taking surfaces of image taking means 9a to 9e by image pick-up optical systems 13a to 13e, thus obtaining 
taken images Xa to Xe with a parallax. 

[0240] Of these taken images Xa to Xe, only one taken image is observed by the right eye of an observer upon 
observation, and only one taken image is observed by his or her left eye. 

[0241] Therefore, when the taken images Xa to Xe have appropriately set sizes and are taken to have a magnrfica- 
15 tion difference, the "observability- upon displaying the taken images Xa to Xe on an image display apparatus can be 
improved. 

[0242] Fig. 58 is an explanatory view upon displaying images on the basis of the taken images Xa to Xe in an image 
display apparatus for performing stereoscopic display by utilizing a binocular parallax. 

[0243] In Fig. 58, image display means 10 allows to independently observe five display images with a parallax from 
20 the same directions as those in the image pick-up operation. For example, the image display means 10 adopts the len- 
ticular method, large convex lens/large concave mirror method, parallax barrier method, or the like. Display images 1 1 a 
to 1 1e are displayed on the image display means 1 0. 

[0244] The image display means displays the display images 1 1a to 1 1 e corresponding to the taken images Xa to 
Xe, so that the images can be observed from positions on axes a to e. 

25 [0245] The positional relationship between the image display means and the observer is set so that a left eye 6 L of 
the observer is located on the axis e when his or her right eye 6 R is located on the axis c upon observation of the display 
images 1 1a to 1 1e. More specifically, the taken images Xc and Xe are observed as a stereoscopic pair of images, and 
similarly, the taken images Xb and Xd, and Xa and Xc are observed as stereoscopic pairs of images. 
[0246] Therefore, in this embodiment, the taken images Xa to Xe are taken to generate a magnification difference 

30 between each stereoscopic pair of images. 

[0247] For example, when the size of the taken image Xa is assumed to be 100, the sizes of the taken images Xb, 
Xc, Xd, and Xe are respectively set to be 100, 102, 102, and 100, thus obtaining a magnification difference of 2% 
between each stereoscopic pair of images. 

[0248] As practical means for generating the magnification difference, the same means as in the 1 4th embodiment 
35 shown in Figs. 53 and 54 is adopted. For example, as shown in Fig. 59, the magnification correction lens 8 is inserted 
in the optical path of at least one (the image pick-up optical system 13c in Fig. 59) of the image pick-up optical systems 
13a to I3e, thereby appropriately adjusting the synthesized focal length of the image pick-up optical system (13c) and 
the magnification correction lens 8, and the distance from the image pick-up optical system (13c) and the magnification 
correction lens 8 to the image taking means (9c). 
ao [0249] Alternatively, as shown in Fig. 60, at least one (the image pick-up optical system 1 3c in Fig. 60) of the image 
pick-up optical systems 1 3a to 1 3e is constituted as a zoom optical system 88, and the focal length of the zoom optical 
system 88 and the distance from the zoom optical system 88 to the image taking surface of the image taking means 
(9c) are appropriately adjusted to adjust the magnification difference within the range from -3% to +3%. 
[0250] These arrangements allow any stereoscopic pair of taken images to have a desired magnification difference 
45 therebetween. 

[0251] Fig. 61 is a schematic view showing principal part of the 16th embodiment of the present invention. In this 
embodiment, the present invention is applied to an image pick-up apparatus for taking parallax images using a multi- 
lens camera system. The apparatus takes images so that a magnification difference is generated between taken 
images for the right and left eyes of the parallax images. 
so [0252] In Fig. 61 , an image of an object X is taken by cameras Ya to Ye. The image of the object X is formed on the 
image taking surfaces of image taking means 9a to 9e via image pick-up optical systems 13a to 13e, thus obtaining 
taken images Xa to Xe with a parallax. 

[0253] A magnification modulator 1 7 generates a magnification difference by changing the sizes of the taken 
images Xa to Xe by coordinate conversion processing. 
55 [0254] In the coordinate conversion processing of the magnification modulator 1 7, the sizes of the taken images Xa 
to Xe are appropriately varied in such a manner that, if the size of the taken image Xa is assumed to be 1 00, the sizes 
of the taken images Xb, Xc, Xd, and Xe are respectively set to be 101 , 102, 103, and 104, thereby generating magnifi- 
cation differences among the taken images Xa to Xe. 
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[0255] On the other hand, as in the 1 5th embodiment shown in Figs. 57 and 58, if the display position of an image 
display apparatus and the observation position of an observer is known, a magnification difference can be generated 
between corresponding pairs of the taken images Xa to Xe. 

[0256] Furthermore, when the apparatus comprises image display means 1 0 and position detection means 69 as 
5 shown m Fig. 62, the positions of the eyes, 6 R and 6 L , of the observer are detected in real time, and a corresponding 
pair (taken images Xe and Xc in Fig. 62) of taken images Xa to Xe corresponding to display images to be observed are 
selected on the basis of the detected positions of the two eyes 6 R and 6 L , and a magnification modulator 77 can gen- 
erate a magnification difference between the selected taken images (Xe and Xc). 

[0257] Note that the coordinate conversion processing can be attained by the same signal processing as in the 1 1th 
w embodiment. 

[0258] Fig. 63 is a schematic view showing principal part of the 17th embodiment of the present invention In this 
embodiment, input images are generated by synthesis • processing of an image, and generation of input images from 
many virtual view points is allowed using a smaller number of cameras. 

[0259] In this embodiment, a large number of input images with a parallax are generated as in the image pick-up 
15 apparatus using a multi-lens camera system of the 15th embodiment shown in Fig. 57, which obtains a large number 
of taken images with a parallax, and the generated input images are input to an image display apparatus an image 
recording apparatus, or the like. 

[0260] In this embodiment, since all the pieces of image information are converted into signals, and synthesis • 
processing of images is performed by arithmetic operations on a computer, a magnification difference is generated 
20 between right and left input images mainly by the coordinate conversion processing. 

[0261] In Fig. 63, an image of an object X is taken by cameras Ya and Yb. The image of the object X is formed on 
the image taking surfaces of image taking means 9a and 9b via image pick-up optical systems 13a and 13b thus 
obtaining taken images Xa and Xb with a parallax. 

[0262] Arithmetic means 70 performs coordinate conversion processing of the taken images Xa and Xb from the 
25 cameras Ya and Yb as needed to obtain input images Xa' and Xb', and generates an input image Xc'. 

[0263] The arithmetic means 70 generates a taken image Xc, as if it were taken by a virtual camera Yc from the 
direction of a view point c, on the basis of the taken images which are respectively obtained by taking the image of the 
object X by the cameras Ya and Yb from the directions of view points a and b. 

[0264] When the three input images Xa' to Xc' obtained as described above are input to the image display appara- 
tus, and are displayed as display images 20a to 20c, the observer observes only a pair of images for the right and left 
eyes 6 R and 6 L of the display images 20a to 20c at one time. The display images 20a to 20c to be observed at this time 
will be explained below with reference to Fig. 64. 

[0265] Fig. 64 is an explanatory view when the input images Xa' to Xc' obtained in this embodiment are input to and 
displayed by the image display means. 

[0266] In Fig. 64, image display means 20 allows to independently observe three display images from the same 
directions as those in the image pick-up operation. For example, the image display means 20 adopts the lenticular 
method, large convex lens/large concave mirror method, parallax barrier method, or the like. 

[0267] The image display means 20 displays the display images 20a to 20c corresponding to the input images Xa* 
to XC so that the images 20a to 20c can be observed on axes a to c. The image display means 20 sets the positions of 
those images, so that the left eye 6 L of the observer is located on the axis c when the right eye 6 R is located on the axis 
a. 

[0268] More specifically, the observer observes the display images Xa and Xc, and the display images Xc and Xb 
as stereoscopic pairs of images. 

[0269] Therefore, when such an image display apparatus is used, a magnification difference can be generated 

45 between the stereoscopic pair of images 20a and 20c, or 20c and 20b. 

[0270] When the positional relationship between the two eyes 6 R and 6 L of the observer and the image display 
apparatus is unknown, the positions of the two eyes of the observer are detected in real time, and a magnification dif- 
ference is generated between two input images corresponding to the positions of the two eyes of the observer by the 
image coordinate conversion processing on a computer (arithmetic means 70) in correspondence with the positional 

so relationship relative to the image display apparatus, thus practicing the present invention. 

[0271] Figs. 65A and 65B are schematic views showing principal part of the 1 8th embodiment of the present inven- 
tion. In this embodiment, input images having a parallax from arbitrary view points are generated on the basis of single 
three-dimensional information by a CG (computer graphics) technique, and are input to an image display apparatus In 
this apparatus, a magnification difference is generated between input images to be presented to the right and left eyes 

55 of the input images having a parallax. 

[0272] In Fig. 65A, an image memory 80 stores three-dimensional information. Arithmetic means 81 generates a 
plurality of input images having a parallax on the basis of the three-dimensional information in the image memory 80. 
[0273] A magnification modulator 82 performs coordinate conversion processing of an input image from the arith- 
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metic means 81 to generate a magnification difference. 

[0274] In Fig. 65A, the arithmetic means 81 obtains input images Xa" to Xe M corresponding to taken images Xa to 
Xe with a parallax, which are taken by a multi-lens camera system shown in, e.g., Fig. 56 from five directions a to e. 
[0275] When the positional relationship between the right and left eyes 6 R and 6 L of the observer and the image 
5 display means is known, at least one of two input images, corresponding to the right and left eyes of the observer, of 
the input images Xa" to Xe" is subjected to coordinate conversion processing, thereby generating a magnification dif- 
ference. 

[0276] On the other hand, when the positional relationship between the two eyes of the observer and the image dis- 
play means is unknown, a detector 84 is arranged, as shown in Fig. 65B to detect the positions of the right and left eyes 
io 6 R and 6 L of the observer with respect to image display means 83 in real time, and a magnification difference is gener- 
ated between the two input images corresponding to the right and left eyes 6 R and 6 L of the observer, thus practicing 
the present invention. 

[0277] In this embodiment, as described above, since a magnification difference is generated between images to 
be presented to the right and left eyes of the observer of a plurality of input images with a parallax, the "observability" 
75 upon displaying images on the image display apparatus is improved. 

[0278] Note that the magnification difference is preferably set to fall within the range of 0 < Imagnification differencel 
< 3 from Figs. 42A and 42B. 

[0279] Furthermore, in order to set a value effective for many users, the magnification is preferably set to be about 

2%. 

20 [0280] Fig. 66 is an explanatory view of the 1 9th embodiment of the present invention, and Fig. 67 is a schematic 
view showing principal part of this embodiment. 

[0281] Fig. 66 illustrates an object X, color filters 181 n and 181 L for the right and left eyes, and eyes 6 R and 6 L of 
the observer. Steps (66-a) to (66-d) will be described below. 

25 (66-a) Two input images with a parallax are obtained by the double-lens camera method, image synthesis* 

processing method, CG method, or the like. 
(66-b) Two input images are synthesized to one (display image). 
(66-c) The display image is displayed in two colors. 

(66-d) The display image is observed via the color filters 1 81 R and 1 81 L for the right and left eyes. 

30 

[0282] In order to give a magnification difference to the display image for the right or left eye, one of the following 
methods is used. 



(1) The asymmetrical optical system method or electronic image coordinate conversion processing is executed in 
35 step (66-a). 

(2) The image coordinate conversion processing is executed in step (66-b). 

(3) The color filters 1 81 R and 1 81 L are asymmetrically arranged in step (66-d). 

[0283] When two input images with a parallax are projected onto a single screen 192 using two projectors 1 91 R and 
40 1 91 L with color filters, as shown in Fig 67, the input images need not be synthesized to one image in advance, and the 
procedure can be modified to the one shown in Fig. 68. This procedure will be described below. 

(68-a) Two input images with a parallax are obtained by the double-lens camera method, image synthesis • 
processing method, CG method, or the like. 
45 (68-b) Two images are projected using the two projectors 191 R and 191 L onto the screen 1 92 via the color filters 

on the basis of the two images, thus superimpose-displaying the images as display images. 
(68-c) The display images are observed via the color filters 1 81 R and 1 81 L for the right and left eyes. 

[0284] In this case, in addition to the above-mentioned methods, the projection optical systems of the two projectors 
so for projecting the display images may be asymmetrically set to generate a magnification difference. 

[0285] Fig. 69 is a schematic view showing principal part of the 20th embodiment of the present invention. This 
embodiment is based on the polarizing glass method in which display images for the right and left eyes are displayed 
in different polarizing states, and are observed via polarizing glasses. 

[0286] In Fig. 69, lenses 21 1 R and 21 1 L for the right and left eyes of polarizing glasses 21 1 have polarizing filters 
55 with different polarizing directions. 

[0287] The general procedure of stereoscopic image display by the polarizing glass method will be explained below 

(69-a) Two input images with a parallax are obtained by the double-lens camera method, image synthesis • 
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processing method, CG method, or the like. 

(69-b) The two input images are displayed as display images on two image display means. 

(69-c) Polarizing plates are arranged on the respective image display means, and the two display images are 
superimpose-displayed using a half mirror. 

(69-d) The display images are observed via the polarizing glasses. 

[0288] In this embodiment, in order to generate a magnification difference between the display images for the right 
and left eyes, one of the following methods is used. 

(1) The asymmetrical optical system method or electronic image coordinate conversion processinq is used in step 
<69-a). K 

(2) The image coordinate conversion processing is used in step (69-b). 

(3) The optical systems for projecting the display images for the right and left eyes are asymmetrical ly set in step 
(69-c). 

is (4) The lenses 21 1 R and 21 1 L for the right and left eyes of the polarizing glasses 21 1 are asymmetrically set in step 

(69-d). 

[0289] Fig. 70 is an explanatory view of the 21st embodiment of the present invention. This embodiment is based 
on the time sharing shutter method in which display images for the right and left eyes are displayed in a time sharing 
20 manner, and are observed via time sharing shutter glasses. 

[0290] In Fig. 70, time sharing shutter glasses 221 have lenses 221 R and 221 L for the right and left eyes, which are 
synchronized with the time sharing display operations of the display images. 

[0291] The generation procedure of stereoscopic image display by the time sharing shutter method will be 
described below. 



25 
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(70-a) Two input images with a parallax are obtained by the double-lens camera method, image synthesis- 
processing method, CG method, or the like. 

(70-b) The two input images are displayed as display images on image display means in a time sharing manner. 
(70-c) The display images are observed via the time sharing shutter glasses. 

[0292] In this embodiment, in order to generate a magnification difference between the display images for the right 
and left eyes, at least one of the following methods is used. 

(1) The asymmetrical optical system method or electronic image coordinate conversion method is used in step (70- 
35 a). 

(2) The image coordinate conversion method is used in step (70-b). 

(3) The lenses 221 R and 221 L for the right and left eyes are asymmetrically set in step (70-c). 

[0293] Fig. 71 is an explanatory view of the 22nd embodiment of the present invention. In this embodiment, the 
40 present invention is applied to an image display apparatus of the large convex lens/large concave mirror method. ' 
[0294] The general procedure of the large convex lens/large concave mirror method will be described below. 

(71 -a) n input images with a parallax are input by the multi-lens camera method, image synthesis • processing 
method, CG method, or the like. 
45 (71 -b) The n input images are simultaneously projected as display images at positions near the large convex lens 

or large concave mirror using n projectors. 

(71 -c) Two out of the plurality of display images projected to the positions near the large convex lens or the large 
concave mirror are observed as a stereoscopic pair of images (n is a natural number). 

so [0295] In this embodiment, the position where the observer observes with respect to the large convex lens or the 
large concave mirror (to be simply referred to as an observation position hereinafter) is specified in advance, and a 
magnification difference is generated between a stereoscopic pair of images observed from the observation position. 
[0296] On the other hand, the actual observation positions of the observer may be assumed without specifying the 
observation position, and the sizes of the display images may be appropriately set as in the 17th embodiment shown in 

55 Fig. 61. 

[0297] Furthermore, as shown in Fig. 62 of the 17th embodiment, the detector 69 may be arranged to detect the 
positions of the right and left eyes 6 R and 6 L the observer, and a stereoscopic pair of images to be observed may be 
selected in correspondence with the detected positions, thus generating a magnification difference therebetween. 
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[0298] In this embodiment, in order to generate a magnification difference between the stereoscopic pair of images, 
at least one of the following methods is used 

(1 ) The asymmetrical optical system method or electronic image coordinate conversion method is used in step (71 - 
sa). 

(2) The image coordinate conversion method is used in step (71 -b). 

[0299] Fig. 72 is an explanatory view of the 23rd embodiment of the present invention. In this embodiment, the 
present invention is applied to an image display apparatus of the lenticular method. 
io [0300] The general procedure of the lenticular method will be explained below. 

[0301] Fig. 72 illustrates a lenticular lens 241. A display device 242 comprises, e.g., a liquid crystal panel or a CRT. 
[0302] The procedure of the lenticular method will be described below. 

(72-a) n input images with a parallax are input by the multi-lens camera method, image synthesis • processing 
15 method, CG method, or the like. 

(72-b) The width of each input image is compressed to 1/n, the compressed image is horizontally divided into n por- 
tions, and divided pixels are picked up and arranged (synthesized) one by one from the n input images, thus obtain- 
ing n synthesized display images. 

(72 -c) The n display images are displayed on the display surface of the display device 242. 
20 (72-d) Two out of the n display images are observed as a stereoscopic pair of images via the lenticular lens 241 (n 

is a natural number). 

[0303] In this embodiment, in order to generate a magnification difference between the stereoscopic pair of images, 
at least one of the following methods is used. 

25 

(1) The asymmetrical optical system method or electronic image coordinate conversion method is used in step (72- 
a)- 

(2) The image coordinate conversion method is used in step (72-b). 

30 [0304] In an image display apparatus shown in Fig. 73, n display images are projected onto a transmission type 
double lenticular screen 252 using n projectors 251, and an observer observes the images via the double lenticular 
screen 252. 

[0305] Fig. 74 shows the procedure in this case. The procedure will be explained below. 

35 (74-a) n input images with a parallax are input by the multi-lens camera method, image synthesis • processing 

method, CG method, or the like. 

(74-b) The n input images are projected as n display images onto the double lenticular screen 252 using the n pro- 
jectors. 

(74-c) Two out of the n display images are observed as a stereoscopic pair of images via the double lenticular 
to screen 252 (n is a natural number) . 

CLAUSES 

[0306] 

45 

1. A stereoscopic image display apparatus which allows an observer to recognize a stereoscopic image, compris- 
ing: 

right eye image display means for displaying an image for a right eye by irradiating light; 

so left eye image display means for displaying an image for a left eye by irradiating light; and 

tilt means for tilting a central line, in a vertical direction, of an observation image which is formed based on the 
image for the right eye and is to be observed by the right eye of the observer, and a central line, in the vertical 
direction, of an observation image which is formed based on the image for the left eye and is to be observed 
by the left eye of the observer, relative to each other within a range from an angle larger than 0° to an angle not 

55 more than 3°. 

2. An apparatus according to clause 1, wherein said tilt means comprises means for rotating at least one of said 
right and left eye image display means in a display surface of the image display means. 
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3. An apparatus according to clause 1 , wherein said tilt means comprises an image rotator arranged in an optical 
path of at least one of said right and left eye image display means. 

4. An apparatus according to clause 1 , wherein said tilt means comprises means for rotating a display image of at 
least one of said right and left eye image display means by coordinate conversion processing, and displaying the 
rotated image on the image display means. 

5. An apparatus according to clause 1 , wherein said tilt means comprises one of means for rotating clockwise the 
central line, in the vertical direction, of the observation image to be observed by the right eye of the observer and 
means for rotating counterclockwise the central line, in the vertical direction, of the observation image to be 
observed by the left eye of the observer. 

6. An image pick-up apparatus for taking an image which is used to allow an observer to recognize a stereoscopic 
image, comprising: 

right eye image pick-up means for taking an input image for a right eye of the observer, and taking the input 
image; J h 

left eye image pick-up means for taking an input image for a left eye of the observer, and taking the input image- 
and 

rotation means for rotating at least one of said right and left eye image pick-up means in a pick-up surface of 
the image pick-up means so as to tilt and take the input images for the right and left eyes relative to each other. 

7. An apparatus according to clause 6, wherein said rotation means comprises an image rotator arranged in an 
optical path upon taking one of the input images for the right and left eyes. 

8. A stereoscopic image display apparatus which allows an observer to recognize a stereoscopic image, compris- 
ing: 

right eye image display means for displaying an image for a right eye by irradiating light; 

left eye image display means for displaying an image for a left eye by irradiating light; and 

means for setting a size of an observation image which is formed based on the image for the right eye and is 

to be observed by the right eye of the observer, and a size of an observation image which is formed based on 

the image for the left eye and is to be observed by the left eye of the observer to be relatively different from 

each other. 

9. An apparatus according to clause 8, wherein said means for setting the sizes of the observation images to be 
relatively different from each other comprises means for changing a size of a display image of at least one of said 
right and left eye image display means by coordinate conversion means, and displaying the image of the changed 
size on the image display means. 

10. An apparatus according to clause 8, further comprising: 

a right eye optical system for forming the observation image to be observed by the right eye of the observer on 
the basis of the image for the right eye; and 

a left eye optical system for forming the observation image to be observed by the left eye of the observer on 
the basis of the image for the left eye, and 

wherein a focal length of said, right eye optical system is different from a focal length of said left eye optical sys- 
tem. 

1 1 . An apparatus according to clause 1 0, wherein the focal length of at least one of said right and left eye optical 
systems is variable. 

12. An apparatus according to clause 8, wherein said apparatus satisfies: 

0 < IAVI < 3 

where R v and are the sizes of the observation images to be observed by the right and left eyes of the observer, 
and AV is the magnification difference given by: 



28 



1 083454A2 I > 



EP 1 083 454 A2 



AV = (R V -L v yi_/100 

13. An image pick-up apparatus for taking an image which is used to allow an observer to recognize a stereoscopic 
image, comprising: 

a right eye image pick-up optical system for taking an image for a right eye of the observer; and 
a left eye image pick-up optical system for taking an image for a left eye of the observer, 
wherein the right and left eye image pick-up optical systems have different magnification factors. 

14. An apparatus according to clause 1 3, wherein the magnification factor of at least one of said right left eye image 
pick-up optical systems is variable. 

15. An apparatus according to clause 13, wherein said apparatus satisfies: 

0 < IAVI < 3 

where AV is the magnification difference between said right and left image pick-up optical systems. 
Claims 

1 . A stereoscopic image display apparatus which allows an observer to recognize a stereoscopic image, comprising: 

right eye image display means for displaying an image for a right eye by irradiating light; 

left eye image display means for displaying an image for a left eye by irradiating light; and 

means for setting a size of an observation image which is formed based on the image for the right eye and is 

to be observed by the right eye of the observer, and a size of an observation image which is formed based on 

the image for the left eye and is to be observed by the left eye of the observer to be relatively different from 

each other. 

2. An apparatus according to claim 1 , wherein said means for setting the sizes of the observation images to be rela- 
tively different from each other comprises means for changing a size of a display image of at least one of said right 
and left eye image display means by coordinate conversion means, and displaying the image of the changed size 
on the image display means. 

3. An apparatus according to claim 1 , further comprising: 

a right eye optical system for forming the observation image to be observed by the right eye of the observer on 
the basis of the image for the right eye; and 

a left eye optical system for forming the observation image to be observed by the left eye of the observer on 
the basis of the image for the left eye, and 

wherein a focal length of said right eye optical system is different from a focal length of said left eye optical sys- 
tem. 

4. An apparatus according to claim 3, wherein the focal length of at least one of said right and left eye optical systems 
is variable. 

5. An apparatus according to claim 1 , wherein said apparatus satisfies: 

0 < IAVI < 3 

where R v and Ly are the sizes of the observation images to be observed by the right and left eyes of the observer, 
and AV is the magnification difference given by: 

AV = (R V -L V )/L/100 

6. An image pick-up apparatus for taking an image which is used to allow an observer to recognize a stereoscopic 
image, comprising: 
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a right eye image pick-up optical system for taking an image for a right eye of the observer; and 
a left eye image pick-up optical system for taking an image for a left eye of the observer, 
wherein the right and left eye image pick-up optical systems have different magnification factors. 

5 7. An apparatus according to claim 6, wherein the magnification factor of at least one of said right left eye image pick- 
up optical systems is variable. 

8. An apparatus according to claim 6, wherein said apparatus satisfies: 

10 0 < f AVI < 3 

where is the magnification difference between said right and left image pick-up optical systems. 
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